Research

A comprehensive study on developing large language models for global medical applications, with a focus on multilingual capabilities including African languages.

MORENA is a 1.5-billion-parameter decoder trained from scratch, with its own tokenizer and data mixture, for twelve Latin-script African languages plus English, French and code. On held-out African text the base model reaches 1.408 bits per byte, the lowest of 26 models measured, including the 8B African specialist Lugha-Llama-8B and gemma-3-12b-it. Prompted with three examples, the instruct model translates English into five African languages at chrF++ 45.8, eight points above MADLAD-400-3B.