- Windows Copilot Runtime is a three-layer system on Windows 11 24H2 that combines more than 40 AI models, APIs, and other tools to power AI features, such as Windows Recall, Studio Effects, Auto Super Resolution, and more.
On Windows 11, the Windows Copilot Runtime is a new system that is part of version 24H2 (2024 Update) and higher releases that combine a series of smaller AI models that run on-device and other components to power AI features in the operating system.
What’s the Windows Copilot Runtime?
The Windows Copilot Runtime is a new system that combines a number of components, including AI frameworks, toolchains, the new Windows Copilot Library, and on-device AI models, and it ends with applications and features that provide AI capabilities.
While you may be familiar with the “Copilot” branding on Windows 11, which refers to the chatbot AI feature, the Windows Copilot Runtime is something different. This new system starts immediately above the silicon, which, in this case, refers to the NPUs (Neural Processing Units) and GPUs (Graphical Processing Units), which are the main components to process AI tasks.
Layer 1
In the first layer, you will find the “AI frameworks,” which include DirectML, ONNX Runtime, PyTorch, WebNN, and the “Toolchains,” which include Olive, AI Toolkit for Visual Studio Code, and other tools for developers.
Layer 2
Microsoft is placing the “Windows Copilot Library” and “On-device models” in the second layer. The “Windows Copilot Library” provides a new set of APIs (Application Programming Interfaces) for developers to integrate AI features into their apps.
Also, and more importantly, in this second layer, you will find more than 40 AI models that run simultaneously locally on the computer to power many of the new AI experiences, such as (the controversial) Windows Recall, Windows Studio Effects, Auto Super Resolution, Voice Clarity, Cocreator for Paint, and Restyle Image.
As part of the new runtime, Microsoft is integrating the Phi Silica Small Language Model (SLM) and other models, such as the Optical Character Recognizer (OCR), Screen Region Detector, Natural Language Parser, Image Encoder, Vector Embeddings, Retrieval Augmented Generation (RAG), Text Summarization, and User Activity.
Layer 3
Finally, in the third layer, you will find the applications that access these features to deliver AI experiences. However, developers must integrate new AI features by tapping their apps into the Windows Copilot Runtime.
What will happen to Copilot for Windows 11?
Copilot for Windows 11 is now evolving from an integration at the operating system level to a simple web application that connects to the Copilot AI on the web.
This means that Copilot will continue to depend on the cloud to process data and produce responses to your questions. As a result of this change, the chatbot can no longer change system settings, and the “Windows key + C” keyboard shortcut has been scratched.
However, Microsoft is planning to roll out “Copilot suggestions,” which is another evolution of the experience. Instead of asking the chatbot to change settings or perform specific tasks, you will see Copilot suggesting different actions.
For example, in the Settings app, Copilot may suggest to fix a problem or change a setting. On File Explorer, you will be able to access features like the ability to remove the background from an image directly from the context menu.
On a toast notification, Copilot may suggest summarizing a document that arrived in your inbox.
Copilot will also integrate into other apps like Photos and Paint.
While the Copilot app will require a connection to the internet, most of the new AI features will run without an internet connection using the Windows Copilot Runtime.
Can you uninstall the Windows Copilot Runtime?
Unlike the Copilot app, which can be removed, the operating system won’t include an option to uninstall the Windows Copilot Runtime since it’s a system component.
Also, Microsoft isn’t limiting developers to using only the on-device models shipping with the new AI system since developers can bring their own AI models to the operating system to power their applications.
In short, the Windows Copilot Runtime is a new system that uses on-device AI models to power AI features from Microsoft and developers who choose this system. Also, it’s not related to the Copilot app for Windows 11.