Transcript

When building AI applications, one of the biggest challenges isn't how to implement them. It's deciding which model to use. With so many options across providers, sizes and capabilities, it quickly can become overwhelming. In this video, we'll see how the Foundry Toolkit in Visual Studio Code helps you explore, compare and choose the right model for your scenario. Foundry Toolkit gives you access to a wide range of generative AI models, including both small and large language models. You can work with models from sources such as models hosted on Microsoft Foundry, models from providers like OpenAI, Anthropic and Google, models available on GitHub and local models running on your machine using ONNX or Ollama. You can also bring your own models through custom integrations. So instead of being tied to a single provider, you get a flexible multi-model ecosystem directly inside Visual Studio Code. Before exploring the catalog manually, you can start by asking GitHub Copilot for recommendations. For example, you can prompt something like, recommend to models I should compare for a marketing scenario with image input support available in Foundry toolkit and deployable in my Azure subscription and my Azure region. And of course, I specify the name of my Azure subscription, which is AI team, and the Asian region I want the models to be deployed to, which is Sweden Central. Now, GitHub Copilot is looking for the tools it needs to achieve these goals. So to check models which are both available within the Foundry toolkit, available in my Azure subscription and in that specific region, and which also have vision capabilities, so which are able to process images. So it's going to use the Azure MCP server, which exposes tools to check for available models and also check for quotas capabilities in a specific region and a specific subscription. So the final answer is based on my subscription quotas in Sweden Central and Foundry Toolkit Catalog support. Here are two solid vision capable models which are GPT 5 and Cloud Zone at 4.6. So I have a nice table in here with comparative analysis between the two models. and both accept prompt photos and mood board images as input and return marketing copy. It also gives me some quick alternatives like GPT-5 and GPT-4O or Claude Sonnet and Claude Haiku. And it also gives me some next steps to compare these models and test different prompts across the two models in the Foundry Toolkit model catalog. So among the recommendations, all the recommendations that GitHub Copilot just share with me, I would go with this GPT-5 versus GPT-4O comparison as I want to compare models of the same provider of OpenAI. So now if I go back to the Foundry toolkit extension and I opened the model catalog among the available developer tools under the discover umbrella. I can kind of validate GitHub Copilot recommendations and further explore the catalog. So the model catalog is our central place to discover and explore models. At the top, we can browse popular models across different categories. We can search by name or description. We can explore models from multiple providers like Microsoft Foundry. We also have GitHub hosted models. We have local models and so on and so forth. We can also narrow down our search by using some filters here. For example, since we know we want to deploy our models to Microsoft Foundry, I'm going to select hosted by Foundry as first selection. And then the second filter is about the publisher. And since we want to compare GPT-5 to GPT-4O, I'm going to select OpenAI as publisher in here. And then in terms of feature, we can select image attachment because we know we want for our marketing scenario, we want to be able to process image as input. And then there are a couple of other filters we are not going to use, which are model type, in case you want to narrow down your search based on local run with CPU, GPU, or NPU, or if you want fine-tunable models. So we have kind of narrowed down our list to 39 models. But since I've got some specific recommendations from Gita Copilot, I can also simply write the name of the model in here. So OpenAI GPT 5, for example. And I can also look at the model card of the model, which basically collects the details of the models given by the model provider. Things like key key use cases, pricing, technical specs, and so on and so forth. Let's deploy OpenAI GPT-5 into our Microsoft Foundry project. And to do so, we can do everything within this Foundry toolkit UI. We can customize the deployment name if we want. We can customize the tokens per minute rate. I'm going to increase these a bit, and then I'm going to click to deploy to Microsoft Foundry. Okay, I've deployed successfully my first model. Now let's go to deploy our GPT-4O model, which is our second candidate for our marketing business scenario. Same thing here, I can customize the deployment name, I can increase the tokens limit a bit, and I can deploy my model to Microsoft Foundry. Nice. So both models have been deployed in my project. So now if I go to the Foundry toolkit nav bar, in my resources, under my project FDK-demo I've created, I've got my two models, GPT-5 and GPT-4O instantiated. And what I can do is I can open one of those in the playground. So I load GPT-5 via Microsoft Foundry in here in the playground. So I can test prompts over my model. But since I want to compare this with the other candidate first, I can click on compare and then select GPT for all via Microsoft Foundry as comparison. So here I get basically a compare view playground where I can test the same prompt against the two different models and see how they behave. For example, let's start with a simple prompt, like create a short linking post about developer productivity with AI tools. So you can see that both GPT-4O and GPT-5 are responding to the same prompt. And you can immediately see the differences. GPT-4O was much faster. It included much more emojis. So it looks more AI generated while we can response might look more human created, more fluent. It is more structured because I have more bullet points. In terms of verbosity, they are pretty similar. Maybe GPT-40 is a little bit more verbose and it included also some hashtags which GPT-5 didn't include. So this looks like more marketing ready than what GPT-5 created. So now let's try another prompt. Let's ask the model to extract the text from the attached image because I want to check how those models are able to process images. So as an example, I'm going to attach an image which is a slide with some GitHub Copilot capabilities and features illustrated. So I'm going to pair these textual an image prompt as input for both models. Again, GPT-4O was much faster. GPT-5 is still generating the response. The extracted text looks quite accurate. create in the IDE, collaborate across the SDLC. And then I also have the text extracted from the footer. So the other thing we can see is that GPT-4O added some tokens to introduce the text, like extracted text, and then the extracted text. And also here I have a specification for the footer, why there's no kind of wasted tokens by GPT-5. But for some use cases, having this kind of separator like extracting text, footer, et cetera, might be useful. So this is the kind of analysis you can do in the compare view of the playground. But let's imagine I'm going to select GPT-40 for this use case, especially because I need my model to respond and react faster. Coming back to the standard playground, beyond comparison, the playground gives you a flexible environment to experiment. So you can select different models in here, you can add a system prompt, An example of system prompt in our case might be something like, you are an intelligent and friendly assistant that supports a social media management team creating content targeted to a developer audience on different channels and formats. And then I have some specification about the role of the assistant. And then I can also adjust some parameters like the max response tokens, the temperature, the top P parameter. And I can iterate over prompts. So to wrap up, Copilot helps you generate an initial shortlist of models. Then the model catalog within the Foundry toolkit allows you to explore and refine your selection. Filters help narrow down the options. And then the playground enables side-by-side comparison and testing. In the next video, we'll go deeper into agent development process.