Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models

arXiv cs.AI · 18 hours ago

Researchers developed a privacy-centric framework for running large language models across edge devices and cloud servers, where edge devices handle preprocessing and partial computation while the cloud performs core inference, with all transmitted data encrypted. The system reduces per-token latency by up to 46.1% and downlink payloads by up to 67.4% compared to baseline split inference approaches. This enables consumer and embedded devices to run LLMs efficiently while protecting user prompts and dialogue data from full exposure to cloud servers.