CoFrGeNets replace the ‘bones’ of transformer-based models
IBM Research
IBM researchers built CoFrGeNets, a new AI architecture that swaps out core transformer parts for math based on continued fractions. It matches or beats models like GPT2-xl using far fewer parameters, and could shrink energy-hungry AI training.
Based on reporting by IBM Research — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Every big-name chatbot right now runs on some flavor of the transformer, the architecture behind GPT, Gemini, Claude, and Llama. It works, but it's expensive: multi-head attention and feed-forward layers pack in huge parameter counts, and that translates directly into heavy compute bills and long training runs. A team at IBM Research decided to poke at the foundation itself rather than tweak around the edges, and the result is something called CoFrGeNet, short for Continued Fraction Generative Networks.
The idea borrows from a centuries-old mathematical trick: continued fractions, nested sequences of divisions that can represent complicated functions with surprisingly compact notation. Amit Dhurandhar, a principal research scientist on the project, said the goal was twofold — give the field an alternative to the small handful of dominant architectural ideas it currently leans on, and probe just how far this particular function class could be pushed. The team had already tried continued fractions on classification tasks with promising results. Extending the concept to generative models, Dhurandhar noted, was a much bigger conceptual and engineering leap.
In practice, CoFrGeNet builds
Read more about this at: IBM Research
