Amália and the Future of European Portuguese LLMs (duarteocarmo.com)

AMÁLIA, a new fully open-source Large Language Model (LLM) dedicated to European Portuguese, has been officially announced following a collaboration among leading Portuguese universities and research labs. The model builds upon the pre-training phase of EuroLLM, enhancing its architecture with modified context length and RoPE scaling. AMÁLIA aims to elevate European Portuguese in the AI landscape, employing targeted data sources throughout its training phases, including 5.8 billion tokens from Arquivo.pt. Despite achieving competitive results against existing models like Qwen 3-8B, questions remain about the proportion of European Portuguese data in its training and the effectiveness of the newly introduced benchmarks for evaluating the model's understanding of Portugal. While AMÁLIA represents a significant step forward for LLMs in European Portuguese, concerns about the openness of its resources have been raised. Unlike other open models such as Olmo, which offers complete access to weights, data, and code, many aspects of AMÁLIA, including model weights and the resulting dataset, are not yet publicly available. The team acknowledges the need for more extensive training data wholly dedicated to European Portuguese to optimize the model further. As development continues, the importance of openness in AI research is highlighted, with a call for transparency to ensure that multilingual and culturally relevant models can thrive and accurately represent their intended languages.