CoFrGeNets replace the ‘bones’ of transformer-based models
IBM Research
IBM Research developed CoFrGeNets, a new neural architecture that replaces transformer components with structures based on continued fractions to reduce parameters and computational overhead. CoFrGeNet models matched or outperformed GPT2-xl and Llama-3.2B while using hundreds of millions fewer parameters and training faster. This architectural approach could be selectively integrated into existing model pipelines to improve efficiency without sacrificing performance.
Why it matters
This new approach, a practical and conceptual shift, points to lighter-weight generative AI models that perform competitively, and in many cases even better.
