Qualcomm will work with Meta to add on-device Llama 2 support for smartphones and PCs

[Inviteshop]

Earlier today we reported on Meta announcing and launching Llama 2, the next-generation version of its large language model for generative AI apps and services. Now, there's word of a new partnership between Meta and Qualcomm that will allow Llama 2 to be used on mobile devices powered by Qualcomm Snapdragon chips.

In a press release, Qualcomm stated the goal was to allow those devices to run Llama 2-based apps and services on those devices, without the need for them to connect to a cloud-based service like other current generative AI products use such as ChatGPT and Bing Chat. Qualcomm stated:

The ability to run generative AI models like Llama 2 on devices such as smartphones, PCs, VR/AR headsets, and vehicles allows developers to save on cloud costs, and to provide users with private, more reliable, and personalized experiences.​

Qualcomm says the ability to run large language models like Llama 2 on a device has a number of advantages. It could be more cost-efficient versus using a cloud-based LLM and offer better performance, as it won't have to connect to an online service. At the same time, an on-device LLM can offer more personalization of AI services. It can also be more secure and private compared to connecting to a cloud server.

Currently, Qualcomm plans to start supporting Llama 2-based AI services on devices that use Snapdragon chips sometime in 2024. There's no word yet on if it will need the latest generation Qualcomm chips to work, or if that kind of support can be made backwards compatible with current Snapdragon chips.

Meta says that Llama 2 has been trained on 40 percent more data than the first-generation Llama LLM. It already announced a partnership with Microsoft to make Llama 2 available for free to commercial and research users on its Azure services, along with a way to download and run the LLM locally on Windows PCs.