Liquid AI Open-Sources Pipette: A Reproducible Benchmarking Suite That Measures On-Device Models, Quantization, Runtime and Hardware Together
MarkTechPost 1 month ago 34
Liquid AI released Pipette, an open-source benchmarking platform that evaluates foundation models on edge devices using a deployment-specific unit (model plus quantization plus runtime plus device) validated in partnership with Artificial Analysis. At 4,096 tokens, two 350M models at the same quantization on the same phone retained 78.4% vs 33.8% of decode throughput. By shifting measurement from model-in-isolation to full system configurations, Pipette enables reproducible on-device comparisons and publishes verified benchmark results via an Apache 2.0 stack and apps.