AI Optimization Blog Post: 2. Fine-tuning AI/LLM models in Azure AI Foundry

Select an Azure AI Foundry project in Azure Portal or make a new one with the "Create a resource" button or use the search field to look for the "Azure AI Foundry" item.

Note: If you log in for the first time or you did not interact with any items before, the "Recent" list will be empty!

After choosing or creating an Azure AI Foundry project, on this screen select "Launch studio".

On this newly-opened webpage you can see the details of the project. In the left side of the screen you can see a menu with different options.

Select "Fine-tuning".

Select "Fine-tune model".

Note: If you did not fine-tune a model before, the list will be empty!

Azure AI Foundry offers several base-models.

You can choose one as a starting point and fine-tune it with custom data.

Select a model from the list.

Here you can see details of the selected model.

Select "Next".

Here you can select "Customize" and change the resource if needed, if not select "Confirm".

Here you can select a version of the model and append a custom suffix.

You also have to choose a resource.

Select "Next".

Note: Version selection is not applicable for all models and appending a suffix is optional!

Here you can specify the location of the fine-tuning data.

Select "Next".

Optional: Here you can select the validation data for the fine-tuning.

Select "Next".

Optional: Here you can customize the fine-tuning parameters if needed, if not select "Next".

Here you can review and modify all the settings and customizations you made once again before beginning with the the fine-tuning.

If you are ready to start the fine-tuning of the model, select "Submit".

Note: This will initiate the fine-tuning process!!!

Reviewing the fine-tuning process

After the fine-tuning process has ended you can see the fine-tuned model listed in the "Fine-tuning" tab.

If you select a newly fine-tuned model in the list you will be able to see all the relevant details about it.

Here you can see all the details related to the fine-tuning and the results.

Select "Metrics" if you want to see more details.

Here you can see a graphical representation of the "train_loss" and "train_mean_token_accuracy" values.

• train_loss: This metric represents the loss value calculated on the training dataset. It indicates how well the model is performing during training. A lower loss value generally signifies a better fit of the model to the training data.
• train_mean_token_accuracy: This metric measures the average accuracy of the model in predicting individual tokens (e.g., words or characters) correctly during training. Higher accuracy indicates that the model is making more correct predictions.

Select "Logs" to see more details.

Here you can see the exact details of the fine-tuning.

This is the beginning of the logs.

You can navigate through the logs with "Prev" and "Next".

After navigating to the latest logs you can see how many tokens have been billed by Azure AI Foundry.

Deployment

Note: Deploying a custom model implies extra hourly costs compared to base models!!!

More info about pricing.

The custom model is now ready for deployment!

Select "Models + endpoints".

Here you can see all the deployed models.

Select "Deploy model".

From the drop-down menu select "Deploy fine-tuned model".

Here you can deploy the freshly fine-tuned model.

Select "Submit".

Note: After deploying the model you can see it added in the "Models + endpoints" tab's list!

The model is now ready to be used!

Select the "Playgrounds" tab.

Select "Try the Chat playground".

Here you can select the fine-tuned model from the "Deployment" drop-down menu.

You can start the chat with the model right away!

What's next?

On 7/14/2025 we'll continue our guide how to use language while communicating with AI/LLM models.