Self-Driving Cars Could Someday Take Requests
IEEE Spectrum Edd Gent
Researchers made a self-driving car listen to plain-English requests like “go fast.” It still stays inside a safe motion planner, so the chatty part can’t take over.
Based on reporting by IEEE Spectrum, Edd Gent — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
A team at Delft University of Technology thinks passengers may someday “talk” to a self-driving car the way they already nag human drivers. The trick is not to hand over control. It is to let language shape how the car drives: faster, smoother, more cautious, depending on what someone asks for.
The researchers built a system that uses a large language model to turn natural-language requests into changes to a motion planner, the software that chooses a safe and efficient route through traffic. Their version keeps the planner in charge and uses OpenAI’s GPT-4o-mini to interpret the passenger’s intent. They presented the work at the IEEE Intelligent Transportation Systems Conference in September and posted a preprint on arXiv.
That separation matters. Instead of letting the model steer the vehicle directly, the system uses the model to rank how important different driving criteria should be, such as speed, steering angle, and collision probability. Those rankings then nudge a model predictive-path integral controller within a safe baseline set by the researchers. If someone says they feel dizzy, the system can shift toward gentler acceleration and smoother turns. If they say they are running late, it can push the car toward a more urgent style.
Before anything changes, the system explains its plan in plain language and asks the passenger to confirm. It also stays interactive, so the user can keep adjusting things if the car’s behavior does not match what they meant. That is the point of the whole setup: reduce the gap between vague human preferences and the hard-edged logic of a driving controller, without pretending that a passenger should be allowed to poke at safety systems directly.
In simulation, using the nuPlan self-driving environment and highway-merging scenarios, the system responded to eight different prompts in ways that matched the intended style. Comfort requests made the ride smoother. Urgent requests raised speed. The researchers say the approach could eventually plug into a car’s perception system, replacing the handwritten scene descriptions they used for the study.
Others in the field see the same basic trade-off. Nicolas Baumann at ETH Zurich has worked on a similar idea for a racing-car controller, and says the value comes from keeping the LLM away from direct control so a hallucination cannot turn into a crash. Matthias Althoff at the Technical University of Munich goes further: his group adds a mathematical safety check before any LLM-suggested decision is used. Delft’s setup is useful, but it is not that strict yet.
My take — AI-written commentary, not fact-checked reporting
This is the sane version of the self-driving-AI pitch: let the model talk, not drive. The industry keeps confusing “natural language” with “permission to improvise,” and that usually ends the same way—someone in a demo smiles right up until the safety team turns pale.
Read more about this at: IEEE Spectrum