Kompact AI by Ziroh Labs and IIT Madras: Making Powerful AI Run Smoothly on Regular Computers

In a major breakthrough for affordable and accessible artificial intelligence, Indian startup Ziroh Labs has joined hands with the Indian Institute of Technology Madras (IIT Madras) to launch Kompact AI — a new AI framework that can run powerful AI models using standard CPUs, without the need for expensive and scarce GPUs.

Artificial intelligence systems typically rely on high-end graphics processing units (GPUs) like those produced by Nvidia to function at scale. These GPUs are not only extremely costly but are also in limited supply, making it hard for smaller companies, researchers, and developing nations to keep pace with the AI revolution. Kompact AI aims to break that barrier.

What makes this development significant is that Kompact AI allows large AI models to operate on central processing units (CPUs) — the kind of chips found in everyday laptops and desktops. This not only drastically reduces the hardware cost for running AI but also makes AI deployment possible in places where access to GPUs is limited or non-existent.

The system mainly focuses on AI inference — the stage where trained AI models are used for tasks such as answering questions, generating text, or identifying images. In a live demonstration, Ziroh Labs showed Kompact AI running Meta’s Llama 2 and Alibaba’s Qwen2.5 models on a regular laptop powered by an off-the-shelf Intel Xeon processor. The results were not only efficient but also of high quality.

This move is in line with a growing trend among AI developers globally who are trying to reduce their reliance on expensive GPU clusters. After China’s DeepSeek showed it could train and run advanced AI models at a fraction of the cost, efficiency-focused AI solutions are gaining momentum. Ziroh Labs is now at the forefront of this wave in India.

Kompact AI has already attracted attention from global chip giants like Intel and AMD, who have tested the framework themselves. The startup believes this innovation could have a major impact in the coming years, making AI development more democratic and accessible.

According to V. Kamakoti, Director of IIT Madras, the current AI landscape favors only those with deep pockets and top-tier hardware. “We’re demonstrating that you don’t need a revolver to kill a mosquito,” he said, emphasizing how even regular CPUs can now handle AI tasks efficiently.

Industry veteran William Raduchel, former Chief Strategy Officer at Sun Microsystems and now a tech adviser to Ziroh Labs, also called this innovation a potential game-changer. He noted that such CPU-friendly AI platforms could shift the balance in global AI development.

In a country like India, where access to high-end GPUs is often limited due to cost and availability, Kompact AI could bridge the “AI divide” by opening doors for students, startups, and institutions to work with advanced AI without heavy investments. From healthcare to education and local industry, the possibilities for real-world application are immense.

With this development, Ziroh Labs and IIT Madras are not just solving a technical problem — they’re pushing the AI movement towards a more inclusive and cost-effective future.