Generative AI already has made a significant mark with people around the world as they use it for many tasks — composing emails, synthesising complex topics, producing photo-realistic images for print, and even creating new music. It’s clear that consumers have flexed their creativity to apply the technology across a whole range of personal needs.
Generative AIs such as OpenAI’s ChatGPT and DALL-E, Google’s Bard and Midjourney have all become part of our lexicon in the last six months. However, Large Language Models (LLMs) like ChatGPT and Bard, which review enormous volumes of text, identify how words relate to one another, and build models that allow them to reproduce similar text, may have the most impact on organisations in the next few years. From summarising content to drafting new prospect emails, these models hold huge potential for companies and the time has come for them to think about how these newly launched tools will affect and enable their organisations.
But how do organisations get started? Currently, implementing these technologies follows one of two patterns. Which pattern (or both) is chosen for what data and use cases is likely one of the most important decisions an organisation will make in the coming years.
Closed Source SaaS Application Programming Interfaces (APIs)
Companies like OpenAI, Amazon Web Services (AWS) and Google Cloud Platform (GCP) provide public model-as-a-service APIs. All organisations need to start is a small software program, or script, that connects to the API and sends properly formatted requests.
This approach comes with several advantages. Given the low barrier to entry, a junior developer can call an API in just a few minutes. The models behind the API are often the largest and most sophisticated versions available, resulting in fast, in-depth and accurate responses on a wide range of topics.
One example of a company that has made use of an API is Morgan Stanley. Morgan Stanley’s use of GPT-4 to power an internal-facing chatbot for wealth management shows the transformative potential of ChatGPT in organisations. By accessing and synthesising vast amounts of information almost instantaneously, ChatGPT provides employees with valuable knowledge and insights on demand, significantly improving efficiency and client service.
Further, developers are exploiting the API capabilities to extend the tools and even make learning autonomous. AutoGPT, an AI agent developed by Toran Bruce Richards, uses GPT 3.5 and GPT 4 to take a task given in prompt form and continuously break it down into sub-tasks until complete. The excitement of an agent automatically doing work in the background has made this repository released on March 30th, one of the fastest growing in GitHub’s history.
Despite APIs being convenient and powerful, they might not be appropriate for all enterprise applications. Given their public nature, the content of the query will need to be sent to the APIs and may be kept for further development of the model. Concerns over data privacy and use with public large language models have heightened after Samsung employees reportedly leaked sensitive information through ChatGPT, leading to increased scrutiny and bans by data protection authorities. As such, this approach could be unsuitable when handling private data and information.
Open-source models in self-managed environments
Given these limitations, organisations might find value in setting up and running an Open source model by themselves. Organisations can run with on-premises servers or in a cloud environment managed by the organisation.
Open-source models give organisations the ability to explore a variety of models to find the one that works best for them. Leaders can explore the many open-source models available in places like Hugging Face and analyse the pros and cons of each before deciding. Also, Open source models can save money when running smaller models, and remove dependencies on third-party API services.
For example, a Dataiku partner used the Open source BART model in an electronic medical record system to summarise lengthy notes from physicians into short patient histories. The model trained on medical literature and created summaries. Human experts then reviewed the summaries for accuracy and graded overall model quality prior to implementation in any production context.
In contrast to the ease of use of APIs offered by vendors, setting up and keeping an Open source model running can be a complex job, requiring data science and engineering skills. This could end up taking valuable time, should the company have these experts in-house. Additionally, Open source models are typically smaller and more focused on a subject or specific task and thus lack the breadth of topics that public APIs can cover.
Using LLMs responsibly
Regardless of the approach adopted, it is crucial to understand the construction of large language models (LLMs) and ensure their responsible use. Data forms the foundation for training these models, making them susceptible to biases that must be acknowledged or at least remediated. Organisations should also assess the impact on end-users and disclose the extent and manner in which they deployed LLM technologies. Providing guidance to users on interacting with information derived from the model, along with caveats regarding the overall quality and factual accuracy, enables end-users to make informed decisions on interpreting the information.
It is imperative that organisations implement AI governance principles to ensure the proper utilisation of models and maintain an auditable record of technology usage, particularly when problems arise that cause review. AI platforms, such as Dataiku, can function as central control towers in this context.
With generative AI dominating headlines, organisations might be eager to jump into the technology. However, there is often value in a deliberate approach. I would recommend leaders run small-scale experiments and consult teams across the organisation. Finding a few quick wins will identify use cases ready for the technology and unlock generative AI’s potential and the next era of automation.
Grant Case is Regional VP, APJ at Dataiku.
