Engaging consumers in a generative AI world (2023)

In progressgenerative artificial intelligenceThe Internet has reached a turning point. In the near future, virtual assistants with large language models (LLMs) could become general-purpose gateways to the Internet. For business leaders, this means making fundamental decisions about how they interact with consumers.

On the one hand, businesses can leverage APIs like plugins to hand off control of their consumer interfaces to LLM-powered virtual assistants (or other conversational AI). OpenAI's ChatGPT plugin allows consumers to order food and groceries through third-party sites like OpenTable and Instacart; other LLM providers could follow.

Companies, on the other hand, can use individually generated AI models on their own websites and apps to retain control of the interface. Implementations vary, companies choose to build or improve. Bloomberg has developed its own model and plans to incorporate it into its services and features. Expedia has integrated OpenAI models into its own app: users stay on the company's website but use ChatGPT to plan trips.

Both strategies - discard and keep - have advantages and risks. The benefits of one option are often offset by the risks of another. Depending on the specific requirements and risk tolerance, different options can also be beneficial for different use cases. We study these changing market dynamics to gain insights into how each strategy is selected and activated. (See Appendix 1.) Now is the time to do that, before the future is determined by the leaders.

Use third-party platforms

LLM providers create a platform game and offer companies the opportunity to reach customers via the LLM platform. The client interface is a chatbot like ChatGPT; in the future, chatbots could be replaced by powerful virtual assistants. The most apt comparison is WeChat, the Chinese chat app that has grown into a super app and is used by 1.3 billion people every month to access a variety of productsService.1 1 WeChat has become a ubiquitous part of Chinese life. Users can access almost any service they need throughout the day without leaving the UI: paying for food, hailing a cab, shopping online or in-store, using social media, and more. It's like an app store, except that instead of downloading and setting up each individual app, all services are available through "mini-apps" that don't require installation. Comment: 1 WeChat has become a ubiquitous part of Chinese life. Users can access almost any service they need throughout the day without leaving the UI: paying for food, hailing a cab, shopping online or in-store, using social media, and more. It's like an app store, except that instead of downloading and setting up each individual app, all services are available through "mini-apps" that don't require installation. A virtual assistant supported by LLM can develop like an internet portal of your choice.

But in terms of user experience, LLM models can go one step further. In the future, consumers could search for services in a more convenient and automated way. For example, someone planning a vacation can share their preferences (dates, destination, budget, etc.) with the virtual assistant, who creates the itinerary, manages any desired adjustments, and makes the reservation.

Instead of an algorithm sorting through all available options, the virtual assistant chooses which options to show the customer. Virtual assistants can act as “trusted agents” and keep users on the platform, working from search to final transaction without users having to go through the options themselves. This shift will dramatically change the dynamics of SEO and customer retention strategies.

How traditional businesses can benefit

Integrating a remote LLM-powered virtual assistant with a plugin or other API is the fastest and easiest way to reach new customers in the world of Generative AI. Offering services through a platform is a proven way for businesses to easily engage with a large and established customer base that values ​​a variety of services accessible from one location. While conversational AI like chatbots leaves a lot to be desired compared to established platforms like WeChat and Amazon, the novelty of the experience drives customer engagement. This commitment is accelerating at record speed the three powerful flywheels driving the platform's success: scalability, learning, and connectivity. The success of the platform can also influence the success of businesses on the platform. (See Appendix 2.)

economies of scale. Large general-purpose models (which are most likely to be used in virtual assistants due to their broad capabilities and superior conversational skills) are notoriously expensive. (See "Creating state-of-the-art large language models.") However, we expect LLM providers to use theirsFor example, a large user base that would give them valuable economies of scale. This allows companies that want to use virtual assistants to interact with customers to do so without creating a model themselves.

learning outcome. The craze for generative AI is encouraging users to try apps like ChatGPT and Bard. Both types of chatbots benefit from a learning effect (aka direct network effect) of this wave of experimentation: the more people use them, the better. For companies that choose to offer their services through a specific platform, this learning curve offers a significant benefit: you get a great user experience and a top-notch conversational interface.

Equal and cross-page network effects. As more companies join the LLM platform, consumers will find more value and new users will be attracted to the platform (equilateral network effect), which in turn will lead to more companies integrating their services into the platform (cross-border network effect). ). side network effects). These network effects present a significant opportunity for a company to engage with a broad user base and acquire a large number of customers.

Risks of traditional companies

Today, many companies are concerned about the operational risk of using LLM interfaces. For example, the provision of services through an LLM-enabled virtual assistant may result in disclosure of company proprietary information to the LLM provider. However, many of these risks can be mitigated through technology implementation and supplier contracts.

But companies also face strategic risks that they may not be aware of right now. A key risk, intermediary commodization, arises when intermediaries between a company and its customers place less value on the company's unique selling proposition. Like search engines, virtual assistants must prioritize which services to show customers and may receive a commission on sales. The result is typically lower profit margins and service standardization, making brand awareness and promoting quality products more difficult. The As more companies join the platform, the risks increase. The question of how LLM-powered virtual assistants can select (or assist clients in selecting) a company's services or products from a list of common services and products is unclear, leaving companies at greater risk of commercialization.

Over-reliance on third party distribution channels also poses risks. The above example of holiday planning illustrates this risk. When a customer books through a third-party virtual assistant and not through the airline or hotel chain that provides the actual service, the virtual assistant provider can control engagement protocols and service selection patterns, and have a significant impact on customer purchasing behavior. As a result, companies risk losing their direct connection to their customers and the important interaction data that enable them to build brand loyalty and nurture lasting customer relationships.

Engage customers directly through personalization

Organizations with access to valuable, domain-specific proprietary data can double their competitive advantage through their own LLMcustomer experiencewith generative artificial intelligence. The trade-off is mostly in the native user experience, as vendors devote resources to optimizing human engagement compared to LLM-powered virtual assistants. A custom-designed, dedicated model should be user-friendly enough to support the customer's product and encourage customers to keep coming back.

The good news is that many small models, like Alpaca (a 7 billion parameter language model created by Stanford University) and Dolly (a 12 billion parameter language model created by Databricks), are not as cumbersome and expensive to modify as the larger ones virtual models assistant. Using proprietary data to create specialized models, such as models created through fine tuning or retraining, can provide superior performance for specialized tasks. The better the data, the better the model can perform tasks related to the data, although this may come at the expense of language performance.

You can also add functionality and value to the raw data by adding an analysis layer. For example, BloombergGPT, a language model with 50 billion parameters, outperforms general-purpose models for very specific financial tasks such as assessing financial risk.

How traditional businesses can benefit

Organizations that choose to create their own custom experiences can retain exclusive access to their valuable proprietary data and keep it safe. Internal controls give companies more flexibility to create unique features and user experiences without having to rely on another company's technology roadmap. In the case of BloombergGPT, users receive more detailed and accurate financial data, and in return Bloomberg receives more individualized user interaction data that can be used to continuously update their LLM.

If companies have direct access to their customer base, they can use the extensive data from customer loyalty. This allows companies to better understand their customers and build stronger, mutually beneficial relationships. It also strengthens a company's ability to build trust with its customers by providing a sense of security and confidentiality while increasing brand awareness. This is especially valuable for more sensitive interactions like checking bank statements; Consumers often prefer services provided directly by the bank itself.

Risks of traditional companies

The obvious operational risk of this option is simply that investing in in-house capabilities can be prohibitive. But companies don't have to take the most expensive route and build from scratch: They can refine a free, open-source model, or take someone else's model and embed it on their own website.

Executives must also consider less obvious strategic risks. First, they must Meet the requirements to build and maintain top-notch skills in-house. (See "Building State-of-the-Art Language Models at Large Scale.") Professional models must be powerful and easy to use enough to attract and retain customers. But for customers, the definition of "good enough" will evolve as they experience top-of-the-line models and platforms. The data science and engineering talent required to manage these models is currently in short supply.

Additionally, as LLM research becomes more proprietary, the research and development required to maintain a best-in-class model may not be feasible for most organizations. To make matters worse, some top-notch model providers do not allow companies to customize models for their own purposes.

Businesses that choose this option also risk missing out on key customer engagement channels. If companies don't list their services on the popular LLM-based virtual assistant, they run the risk of upsetting their customer base - many of whom have taken to using the assistant instead of visiting the company's website.

Choose and activate your strategy

The world of Generative AI is constantly changing, making it difficult to follow evolving market dynamics. I really want to integrate the LLM plugin today, no problem. For some companies, such as those with small market shares, small customer base, poor data quality or lack of access to robust proprietary data and poor user experience, this will be a given. strategic move.

But with every benefit comes a risk. Companies with a strong customer base and unique products can be better served by maintaining control over the user experience and offering virtual assistant services in-house.

Most companies will likely explore both strategic options. Once they have made their decisions, companies can also take many differentiated operational paths, weighing the risks and benefits of each choice; for example building your own model or integrating a top-notch website with a direct-to-consumer interface or improving a best-in-class (see Appendix 3). We observed this: Bloomberg built a custom model, Salesforce enhanced an OpenAI model to build EinsteinGPT, and Expedia both have ChatGPT (on their website) and ChatGPT integrated into their mobile app.

Executives must also consider several factors that can influence their deployment decisions: customer base and market dominance, high-quality data, talent, and computing power.

Customer base and market advantage

Businesses that choose to go direct-to-consumer (either through build-out or fine-tuning) may not need to make major operational changes, especially if they already have a strong market presence and customer base. However, for companies offering their services through third-party LLM solutions, it is important to keep track of each platform's market progress, understand where their customer base is active, and prevent a lock-in situation to from Changes on the platform to benefit pecking order.

LLM providers are currently competing to deliver a best-in-class user experience and secure a first-mover advantage to attract a key user base that is attractive to businesses. It's unclear who will stand out on the LLM platform, but to protect their market advantage, companies must drive integrations that are portable between platforms and embrace beneficial revenue sharing, data and sharing agreements. protection of intellectual property.

data quality

Unique, high-quality data is a key source of competitive advantage in the world of generative AI. The capabilities of an LLM model depend heavily on the quality and content of the data used to train it. Currently, the data used by these models is primarily text, but over time most models will become multimodal and include data from video, audio, financial transactions, and more. Whether services are delivered through remote assistants or directly to consumers, the best data is key to offering unique services and enhancing the user experience to engage users.

Businesses need to consider all forms of data available to them and use them creatively, especially when they decide to build specialized models. Good data makes it easier to address customers directly and reduces the risk of commercialization by third parties. In cases where companies with high-quality data choose third-party LLM platforms, executives should exercise caution to prevent commercialization.

Talent

When deciding to develop LLM-driven experiences in-house, securing data talent is critical, even with simple tweaks to existing LLMs. Even among data scientists, the skills needed to pre-train and fine-tune for specific purposes are rare. It doesn't help that over the past year, more and more research on building or modifying LLMs has become proprietary and the available research is outdated. This means data scientists need to be retrained to use these models, although there may be limited public information available for retraining.

computer battery

It is technically possible to train and derive LLMs with current CPUs and GPUs. But it will push this hardware to its limits. At the same time, investing in hardware upgrades just to get an LL.M. training may not initially appeal to executives given the cost – a high price to pay for something that happens occasionally.

However, this view does not take into account the additional cost of running LLM on the same hardware. The company pays for every cycle it runs in the cloud and the energy cost for every cycle it runs in its own data center. Custom-facing applications can generate thousands to millions of queries every day, and one million queries can cost as much as $120,000 in computing power. To avoid this snowball effect, executives should consider upfront investments.

Leaders need to act fast to develop their generative AI engagement strategies. On the one hand, working with LLM providers early on will allow traditional companies to better specify contract terms such as revenue sharing and search optimization methods. Additionally, early involvement allows executives to sway the technology roadmap in their favor by negotiating technical features that benefit their organization.

On the other hand, the open source models that exist today are still pretty good. However, as research becomes more proprietary, open source may lag behind. Companies that want to reach out directly to consumers can't wait for that to happen, or for other companies to come up with solutions that can entice customers to buy. Today's turning point is tomorrow's game.

The BCG Henderson Institute is the Boston Consulting Group's strategic think tank dedicated to researching and developing valuable new insights in business, technology, and science through the use of powerful thinking technologies. The institute invites leaders to engage in provocative discussions and experiments to push the boundaries of business theory and practice and implement innovative ideas inside and outside the business world. Visit our institute for more ideas and inspirationwebsiteand follow usLinkedInexiston twitter.

Francois Candelon

Director and Senior Partner; Global Director, BCG Henderson Institute

Paris

Philip Evans

senior advisor

Boston

Leonid Zhukov

Vice President - Data Science

New York

Abhishek Gupta

Responsible AI Senior Solution Delivery Manager

Montreal

Fall Sampantal

lighthouse director

Atlanta - House of Light

Lisa Krell

project manager

Washington, D.C

Gauraf I

Advisor

Neu-Delhi

Sarah Rajendran

Advisor

San Francisco - background