What building Shippy taught us about building agents

Allen Institute (AI2) · 3 days ago

Ai2's Skylight team built Shippy, an AI agent for maritime domain awareness that makes high-stakes operational decisions based on real-time satellite and vessel data. The system uses a modular architecture with a versioned Docker image containing the agent's persona and skills, a deterministic CLI interface to the Skylight API to prevent subtle bugs, and Mothership, a Kubernetes-based hosting platform that isolates each user's session and data with dedicated pods. The team developed a custom evaluation framework scored by subject-matter experts against live data, and Shippy will expand to control Skylight's map interface, route queries to smaller models, and maintain cross-thread memory for persistent user context.