For all the impressive things Large Language Models like ChatGPT, Llama, Gemini, Copilot, Claude, Mistral, Cohere and Grok can do, their power consumption is astronomical (multiple GWh per model!), as is the training of specialist LLMs and even SLMs (small language models) within companies for their own specialist purposes.
Fujitsu is an AI and computing company. Fujitsu Research has 1500 researchers globally that spends as much effort protecting people and the planet from threats AI brings as it does developing cutting edge AI itself.
It has developed a full suite of software to test, monitor and improve AI against bias, ethical issues, and abuse to make it safer for us to use.
It is also developing a low power ARM processor to slash the electricity usage in server farms and supercomputers.
However, the rapid rise of the use of specialist GPU processors for the creation of large language models has exponentially grown processor power consumption in recent years. So we set about trying to reduce this and developed a piece of middleware that can optimise the utilisation of the GPU which is often as low as 50% in typical usage.
Increasing GPU utilisation to near 100% will not noticeably increase its power consumption, but it will enable a given processing task to be done in half the time and half the power usage. It works by being “runtime-ware”. In real time it monitors AI framework to allocate GPUs and employs advanced scheduling algorithms that optimise job placement and maximise aggregate utilisation. Unlike related products it can do GPU allocation at runtime, deal with memory partitioning and oversubscription together with plug and play integration focusing on GPU efficiency.
Fresh out of the labs, it is now ready for early partners and customers to try it out in a Private Beta. Once you see how good it is, there are many ways we can move forward together from simple licensing to various forms of partnership.
CONTACT: For more information, please contact Fujitsu Research via this link:
https://airtable.com/app5rrc0xuIv7fvSX/pagQtTCr559rlQBtG/form
