TigerBeetle 1000x World Tour

Toolchain Horizons

Exploring
Dependency-Toolchain
Compatibility

github.com/brson/toolchain-horizons

Me and TigerBeetle

Brian Anderson
brson.github.io
TigerBeetle mascot
tigerbeetle.com

I made a
TigerBeetle
Rust client

I got mad about software dependencies.

Frustration
Science experiment

I did an experiment about it.

TigerBeetle Clients

Python
Java
Go
Node.js
.NET
Rust
Zig

The TigerBeetle Rust Client

  • Small — < 2 kloc.
  • async / await
  • FFI — mostly unsafe.
  • Partially code-generated. 
pub fn lookup_transfers(
    &self,
    events: &[u128],
) -> impl Future<...> {
    let (packet, rx) = create_packet(
        TB_OPERATION_LOOKUP_TRANSFERS,
        events,
    );

    unsafe {
        tb_client_submit(
            self.client,
            Box::into_raw(packet),
        );
    }

    async {
        let msg = rx.await?;
        let responses = handle_message(&msg)?;
        Ok(Vec::from(responses))
    }
}

Establishing a
Minimum Supported Rust Version

"We support
Rust version N"
N = N-1
cargo +N test
Fix build*

Initial MSRV
Rust 1.81
September 5, 2024
yikes

The TigerStyle Mindset

Person meditating

tigerbeetle/docs/TIGER_STYLE.md

Rust Client Dependencies

Cargo.toml


      [dependencies]
      bitflags = "2.6.0"
      futures = "0.3.31"
      thiserror = "2.0.3"

      [build-dependencies]
      anyhow = "1.0.93"
      ignore = "0.4.23"

      [dev-dependencies]
      anyhow = "1.0.93"
      tempfile = "3.15.0"

The Rust futures
Dependency Problem

futures v0.3.31
├── futures-channel v0.3.31
│   ├── futures-core v0.3.31
│   └── futures-sink v0.3.31
├── futures-core v0.3.31
├── futures-executor v0.3.31
│   ├── futures-core v0.3.31
│   ├── futures-task v0.3.31
│   └── futures-util v0.3.31
│       ├── futures-channel v0.3.31 (*)
│       ├── futures-core v0.3.31
│       ├── futures-io v0.3.31
│       ├── futures-macro v0.3.31 (proc-macro)
│       │   ├── proc-macro2 v1.0.90
│       │   │   └── unicode-ident v1.0.14
│       │   ├── quote v1.0.37
│       │   │   └── proc-macro2 v1.0.90 (*)
│       │   └── syn v2.0.88
│       │       ├── proc-macro2 v1.0.90 (*)
│       │       ├── quote v1.0.37 (*)
│       │       └── unicode-ident v1.0.14
│       ├── futures-sink v0.3.31
│       ├── futures-task v0.3.31
│       ├── memchr v2.7.4
│       ├── pin-project-lite v0.2.15
│       ├── pin-utils v0.1.0
│       └── slab v0.4.9
├── futures-io v0.3.31
├── futures-sink v0.3.31
├── futures-task v0.3.31
└── futures-util v0.3.31 (*)

The Experiment

Science experiment animation
  • Pick
    ~30 popular Rust crates.
  • Compile each of them against
    ~every Rust toolchain.
  • Report the toolchain version and date.

Toolchain Horizons: Rust

serdeanyhow
futuressyn
rayonwalkdir
etc.
Rust compatibility timeline

Toolchain Horizons: Rust

serde1.31anyhow1.38
futures1.68syn1.68
rayon1.80walkdir1.31

A TigerBeetle Rust Client
Compatibility Quest!

Hikers on mountain trail

Compatibility Quest Timeline

  • 1.81 (Sep 2024) — Initial MSRV — fs::exists
  • 1.68 (Mar 2023) — futures / syn / proc-macro2
  • 1.56 (Oct 2021) — bitflags, edition 2018
  • 1.51 (Mar 2021) — const generics
  • 1.47 (Oct 2020) — array traits > 32
  • 1.42 (Mar 2020) — matches!, u64::MAX
  • 1.39 (Nov 2019) — async/await minimum

The syn / proc-macro2
epoch

The minimum Rust version
supported by
most of the Rust
ecosystem is limited by the
syn and proc-macro2 crates.

Removing the futures Dependency

Taking out the trash

Step 1: Break Out Sub-crates

Before

[dependencies]
futures = "0.3.31"

After

[dependencies]
futures-channel = "0.3.31"

[dev-dependencies]
futures-executor = "0.3.31"
futures-util = "0.3.31"

Step 2: Polyfill block_on

pub fn block_on<F: Future>(future: F) -> F::Output {
    let mut future = Box::pin(future);
    let parker = Arc::new(Parker::new());
    let waker = parker_into_waker(parker.clone());
    let mut context = Context::from_waker(&waker);

    loop {
        match future.as_mut().poll(&mut context) {
            Poll::Ready(result) => return result,
            Poll::Pending => parker.park(),
        }
    }
}

Step 3: Polyfill Stream Utilities

// Reimplemented from futures-util:
pub trait Stream { ... }           // 8 lines
pub trait StreamExt { ... }        // 12 lines
pub struct Next<'a, S> { ... }     // 15 lines
pub fn unfold<T, F, Fut, Item>     // 45 lines
pub struct Unfold<T, F, Fut>       // 35 lines
macro_rules! pin_mut! { ... }      // 10 lines

Step 4: Polyfill Oneshot Channel

struct OneshotFuture<T> { shared: Arc<OneshotShared<T>> }
struct OneshotShared<T> { waker: Mutex<Option<Waker>>,
                          value: Mutex<Option<T>> }
struct OneshotSender<T> { shared: Arc<OneshotShared<T>> }

impl<T> Future for OneshotFuture<T> { /* 12 lines */ }
impl<T> OneshotSender<T> { fn send(self, value: T) { /* 10 lines */ } }

Should I Even Try?

Does it even matter?

Time Check

Toolchain Horizons

Exploring
Dependency-Toolchain
Compatibility

github.com/brson/toolchain-horizons

Appendix

Toolchain Horizons: Java

Java doesn't seem to have this problem.

Toolchain Horizons: Go

Modules introduced 1.13, September 19.

~60% compatibility.

Go compatibility timeline

Toolchain Horizons

Exploring
Dependency-Toolchain
Compatibility

github.com/brson/toolchain-horizons