Car-GPT: Could LLMs finally make self-driving cars happen?

The Gradient · 2 years ago · 27

Researchers are exploring whether large language models could replace modular approaches to autonomous driving by handling perception, planning, and decision-making tasks like object detection, trajectory prediction, and scene generation. Models such as Talk2BEV and GAIA-1 have demonstrated capabilities in combining vision data with language processing to suggest driving decisions and generate training scenarios. The approach remains experimental with significant concerns about model hallucinations and lack of real-world testing, though proponents argue transparency improvements could eventually address trustworthiness issues.