Generative AI: language models and multimodal foundation models

This report was commissioned by Australia's National Science and Technology Council at the request of the Minister for Industry and Science, the Hon Ed Husic MP in February 2023. The questions asked were: What are the opportunities and risks of applying large language models (LLMs) and multimodal foundation models (MFMs) learning technologies over the next two, five and ten years?; What are some examples of strategies that have been put in place internationally by other advanced economies since the launch of models like ChatGPT to address the potential opportunities and impacts of artificial intelligence (AI)?

ChatGPT is an early example of the kinds of applications and services that will emerge from Generative AI built on LLMs and MFMs. It was developed by the American AI organisation OpenAI - a combined non-profit/for profit entity that received a U$10B investment from Microsoft in 2023. ChatGPT is based on a large language model (LLM) and uses considerable pre- and post-processing of data to deliver a compelling user experience. Given the speed of innovation, quantum of investment and lack of technical information, it is almost impossible to accurately forecast opportunities over the next decade. This report explains how generative AI, based on LLMs and MFMs, currently works, given that the technologies are nascent and rapidly evolving as are the business models, applications and services that are built upon them. Against this backdrop, the report explores foreseeable risks and opportunities, based on current patterns of uptake and application.

Excerpts from publication. Show less