In macOS Tahoe 26.2 an RDMA capability was added for Thunderbolt-5 interfaces. This has been demonstrated to significantly decrease the latency and maintain bandwidth for "clustered" Apple Silicon devices with TB5. What is the ideal and the maximum RDMA burst width for transfers over RDMA-enabled Thunderbolt-5 interfaces?
Ideal and Largest RDMA Burst Width
can you send me test flight code? if possible.
I don’t have time to go through the doc properly today, but we just published a new technote about RDMA over Thunderbolt: TN3205 Low-latency communication with RDMA over Thunderbolt
Please read it through and reply here with your thoughts.
Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"
Quinn, thanks for the quick response. The article that you referenced is excellent. According to the docs, the largest message size via the IB Verbs API is 16,773,120 bytes (~16MB). There doesn't appear to be much performance related data regarding the ideal message size, but I think that with the additional programming notes in the technote, we can port some benchmarks and derive this.
Thanks!
Great thread — RDMA over TB5 is one of the most exciting additions in Tahoe. For anyone looking to benchmark, the IB Verbs API with RDMA Write operations should give the lowest latency path. The ~16MB max message size likely maps to the TB5 link MTU constraints.
It would be interesting to see how the latency profile compares across different burst sizes — especially whether there's a sweet spot below 16MB where you get optimal throughput-per-latency. If you end up running ib_write_bw/ib_write_lat style benchmarks, would love to see the results shared here.