Microsoft Research Blog

Today, we are proud to announce DeepSpeed MoE, a high-performance system that supports massive scale mixture of experts (MoE) models as part of the DeepSpeed optimization library. MoE models are an emerging class of sparsely activated models that have sublinear compute costs with respect to…

Since the computing world began embracing an open-source approach to programming, building software has become increasingly collaborative. Members of development teams with as few as two developers and as many as thousands are simultaneously editing different components in creating software systems and keeping them functioning…

Recently, Transformer-based deep learning models like GPT-3 have been getting a lot of attention in the machine learning world. These models excel at understanding semantic relationships, and they have contributed to large improvements in Microsoft Bing’s search experience and surpassing human performance on the SuperGLUE…

“I’ll put a link to that doc in the chat” “Sorry, my internet is terrible, I’ll put my question in the chat” “That GIF from Amy in the chat is hilarious” If these phrases sound familiar to you, you’re not alone – people the world…

In the pursuit of learning about fundamentals of the natural world, scientists have had success with coming at discoveries from both a bottom-up and top-down approach. Neuroscience is a great example of the former. Spanish anatomist Santiago Ramón y Cajal discovered the neuron in the…

Race and Technology: A Research Lecture Series features 14 distinguished scholars and domain experts from a diverse range of research areas and disciplines. From top left: Dr. Sareeta Amrute, Dr. Kim TallBear, Dr. Charlton McIlwain, Dr. Ruha Benjamin, Dr. Lisa Nakamura, Dr. Simone Browne, and…

Editor’s note: In recent years, telecommunications operators have faced a growing challenge to meet surging global demand for immersive online services and collaboration tools. Upgrading their proprietary networks to prepare for 5G and beyond would require major capital expenditures, even as competition was driving down…

The ability to reason about causality, and ask ``what would happen if…?’’ is one property that sets human intelligence apart from artificial intelligence. Modern AI algorithms perform well on clearly defined pattern recognition tasks but fall short generalizing in the ways that human intelligence can.…

For many years AI researchers have sought to build upon traditional machine learning, which trains technology to process facts and learn from them, and develop machine reasoning, in which programs apply logic to data and solve problems – comparable to the way humans think. For…

The increasing use of personal assistants and messaging applications has spurred interest in building task-oriented dialog systems (or task bots) that can communicate with users through natural language to accomplish a wide range of tasks, such as restaurant booking, weather query, flight booking, or IT…

A Generative adversarial network, or GAN, is one of the most powerful machine learning models proposed by Goodfellow et al. for learning to generate samples from complicated real-world distributions. GANs have sparked millions of applications, ranging from generating realistic images or cartoon characters to text-to-image…

AI agents capable of understanding natural language, communicating, and accomplishing tasks hold the promise of revolutionizing the way we interact with computers in our everyday lives. Text-based games, such as the Zork series, act as testbeds for development of novel learning agents capable of understanding…