Since the launch of ChatGPT a year ago, the landscape of large language models (LLM) has evolved dramatically. We have gone from a model to rule them all to a field with diverse LLMs, each with unique features and capabilities.Â
Alongside the market for proprietary, closed-source models like ChatGPT, an impressive array of open-source LLMs has emerged, matching, and in some cases surpassing, the performance of their private counterparts.
For enterprises developing LLM applications, the argument for leveraging these open-source models is becoming increasingly compelling. The recent controversy surrounding OpenAI further underscores the need for businesses to reassess their LLM strategies and the risks associated with reliance on a single, private model.
Here are four reasons you should consider open-source models for your enterprise.
Transparency
LLMs are often perceived as black boxes, their inner workings still a subject of intense debate. Scientists are divided on whether LLMs truly understand language or are merely rehashing patterns observed in their training data.
This sense of mystery is amplified with closed-source models like ChatGPT. Interacting with such models via an API is akin to dealing with a black box within another black box. User prompts don’t directly reach the model; instead, they traverse a pipeline designed to ensure safety.
The model’s output is also monitored to prevent the generation of unsafe content. Moreover, these APIs and underlying models are regularly updated to counteract new jailbreak prompts and harmful generation.
While this level of control and safety is beneficial for many consumer applications of LLMs, it may not align with the needs of many enterprises. In a business setting, LLMs often serve specific, narrow functions such as document retrieval, writing assistance, or coding support. These applications are typically internal, and enterprises value transparency and stability over a model whose behavior is constantly shifting behind the scenes. For these businesses, an open-source LLM, with its inherent transparency and predictability, may be a more suitable choice.
Control
Closed-source language models like ChatGPT are trained and fine-tuned according to their providers’ policies. While this approach can enhance the models over time, it can also alter their behavior. Consequently, a new version of the model might respond differently to a prompt that it previously handled in a certain way.
Some users have voiced concerns about perceived degradation in the model’s performance. However, this is often not a case of degradation but a change in behavior due to new training data. Furthermore, providers may switch the model operating behind the API for reasons such as cost reduction or improving inference speed, which will also change its behavior.
While these changes can be beneficial for many applications and help individual users achieve their goals more effectively, they can pose challenges when consistency is required. If you need a model to provide the same response to the same prompt consistently, a constantly changing API system may not meet your needs.
Services like the OpenAI API do offer access to specific versions of their model. However, they have a history of phasing out models without providing sufficient warning or allowing ample time for developers to test and adjust their applications with the new models. This lack of predictability can be a significant concern for enterprises that require stability in their LLM applications.
Flexibility
In many scenarios, it is crucial to be able to integrate language models into your existing IT infrastructure. With closed-source models, you’re confined to the API service or the cloud providers that have partnered with the model provider. This limitation can hinder your flexibility and control over your data and models.
Conversely, open-source models offer greater freedom. You can run these models on your own servers or on the cloud provider of your choice. Major cloud providers, including Microsoft and Amazon, have recognized the value of open-source models and offer them in a machine learning as a service (MLaaS) format. Alternatively, you can run your own Docker images on cloud or local servers, ensuring that if you change your infrastructure, your data and models can move with you.
For seamless integration of models into your applications, you can use LLM-serving frameworks like vLLM, TGI, or OpenLLM. These provide a shared interface for interacting with different types of LLMs, simplifying the integration process.
More importantly, the open-source ecosystem is rich with tools that allow you to customize models to your specific needs. For instance, if cost reduction is a priority, you can use various compression and quantization techniques. If personalization is key, cost-effective fine-tuning techniques such as low-rank adaptation (LoRA) and S-LoRA can enable you to run hundreds or even thousands of fine-tuned LLMs at the cost of one.
For tasks involving very long text generation, frameworks like StreamingLLM can extend your model’s context window to millions of tokens without the need for retraining or architectural changes. This flexibility and scalability are unique to the open-source ecosystem and are yet to be matched by the private model market.
Freedom from drama
The recent drama at OpenAI underscores the volatile nature of the AI market. It’s a stark reminder that we’re still navigating the complexities of establishing robust corporate structures for AI companies. If you’re heavily reliant on a closed system like GPT-4, you could be building your applications on a precarious foundation that could crumble unexpectedly.
In contrast, deploying an open-source model gives you complete ownership. It remains unaffected by the politics of its developer, offering a level of stability that closed-source models can’t guarantee. If issues arise with your hosting service, you have the freedom to switch to another cloud provider or server without losing access to your model.
At a minimum, it’s prudent to have a backup plan. An open-source model can serve as a reliable fallback option if a proprietary model fails or is discontinued. This approach ensures continuity of service and mitigates the risks associated with over-reliance on a single, private model.
Long live ChatGPT!
This discussion is not intended to diminish the value of ChatGPT, GPT-4, Claude, or other closed-source models. I continue to use ChatGPT regularly for various tasks, including writing and coding.
Closed-source models will undoubtedly retain their relevance, particularly for single-user applications that don’t have specific integration and customization requirements. However, enterprise applications often have many dependencies and moving parts that must work in harmony. In such environments, private models like ChatGPT can serve as excellent platforms for rapid prototyping and iteration of solutions.
Once your application’s direction becomes clear, however, an open-source model is likely to provide a more robust foundation that can be optimized and improved over time.