Zefiro

In European tradition, a zephyr is a light wind or a west wind, named after Zephyrus, the Greek god or personification of the west wind.

Zefiro is a fine-tuned version of the Mistral model for the Italian language, sponsored by ZefiroAI and an adaptation of the Zephyr model by Huggingface.

Model Details

Zefiro is a porting of the Zephyr model to the Italian language using the wonderful recipes from alignment-handbook . It has also taken inspiration and insights from the Llamantino model developed by Università di Bari. For the implementation we combined different approaches from the two models mentioned but also from the wonderful community of open source.

Model description

• Model type: A 7B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
• Language(s) (NLP): Primarily Italian
• License: Apache 2
• Fine tuned from model: mistralai/Mistral-7B-v0.1
• Developed by: giux78
• Funded by: Zefiro

I have also released a quantized version zefiro-7b-beta-ITA-v0.1-GGUF that can run on CPU based hardware, using fantastic libraries as llama.cpp or various LLM based GUI like Ollama, lm-studio.

In the next release I will try to improve the model using a DPO strategy and release a tuned version of smaller and bigger models like Mixtral and phi-2. I’m also investigating how to evaluate and compare the output models and the different strategies.

Honestly, I’m also looking for sponsored computation (GPU) capacity for training and releasing more datasets and models. If you know someone that can help spread the word …

Contact me if you are interested ale.ercolani@gmail.com