Model Architecture

47 summarised stories about Model Architecture, each linking back to the original source. Browse all topics →

Key players

OpenAI Google MarkTechPost Alibaba Anthropic Jev

Sunday, 13 September 2026

Paper: “Flies are all you need” (on fly-language model)

Artificial Scientific 1 month ago 30 ● 4 sources

The Fly Language Model paper coupled a fixed MaleCNS connectome (as a reservoir) to a frozen 1.17-billion-parameter language backbone and trained only a 278,528-parameter readout. On 32 newly held-out conversations, negative log-likelihood dropped by 0.0222 nats per target token versus the frozen model, but a parameter-matched direct-input readout did slightly better (by +0.000488 nats per token for the fly model). The authors conclude the connectome measurably changes predictions yet does not outperform the simpler matched control, and they release conversational-training code while not releasing the archive or fitted readouts.

Read the full story

Model Evaluation Language Models Model Architecture

FLM (fly-language model): scaling the idea into a fly-based language model

FLM 1 month ago 15 ● 4 sources

FLM (fly-language model) presents a language model idea built using fly neurons. It uses 166,700 fly neurons. The result is a tiny language model that gives imperfect replies once prompted.

Read the full story

Language Models Model Architecture Machine Learning

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

MarkTechPost 1 month ago 31

Yifan Zhang at Princeton proposes a Recurrent Looped Transformer (RLT) that carries the decoder’s final hidden state and layerwise sliding-window attention cache from token t into the next token t+1, with no reset at the prompt–response boundary. The reference design uses 48 encoder and 48 decoder layers, yielding 96 logical blocks per token and a state path depth of 48t after t tokens. The write-up provides an architecture and training/serving specification but reports no measured efficiency, reasoning quality, or scaling results, leaving those validation questions open.

Read the full story

Model Training Language Models Reinforcement Learning Model Architecture

Model Architecture

Key players

Sunday, 13 September 2026

Paper: “Flies are all you need” (on fly-language model)

FLM (fly-language model): scaling the idea into a fly-based language model

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Every AI story that matters, in your inbox by 8am.

Every AI story that matters,
in your inbox by 8am.