loader-logo
Fast track your response to the COVID-19 with chatbot automation. See how we can help.

Are you tracking these success metrics in your enterprise chatbot programme?

chatbot success metrics
The chatbot market has matured significantly in recent years, and now most Chief Digital Officers recognise its role within their end-to-end customer experience strategies.

Share this post

Share on facebook
Share on linkedin
Share on twitter
Share on email

From the early “toy bots” that were essentially a website widget, through to the first incarnations of customer service digital assistants, we are seeing increasing complexity and value created by today’s enterprise chatbot suites.

CDOs are becoming more comfortable delivering their brand experience through chatbots with the industry standards for customer handle rate and resolution. For example, the customer chatbot we built for HSBC US significantly reduced customer contact costs, saved customers 12 minutes per chat, and cut handling time by 50%. CSAT was comparable with human counterparts.

However, one metric that is rarely discussed is the through-life cost of ownership, and the operational agility to be able to update, upgrade and continuously improve the suite of enterprise chatbots. Now that they represent a key part of your IT infrastructure, attention should be given to the internal operations (content specialists, customer analysis, change, deployment and IT integration). 

The internal teams and associated costs can balloon if not properly planned out. Some enterprise chatbots risk becoming monolithic blocks of intents and domains that are a nightmare to navigate, maintain and upgrade. Here are some tips to help prevent this from happening based on our 4 years of working with large enterprises like HSBC, Boehringer Ingelheim, NHS and others:

  1. Make technology choices based on ROI, but consider the investment as the ‘through-life cost of ownership’ rather than just the upfront build cost
  2. Treat chatbot suites like the content management challenge they are, and have a content management strategy and tool (like EBM) at the heart of your strategy
  3. Take a systems approach to the architecture by creating distinct domains of content that can be maintained by the relevant in-house expert and content creator
  4. Incorporate an elegant and appropriate test and release process
  5. Leverage the best of breed NLU engine (Microsoft LUIS, IBM Watson, Google Dialogflow) and recognise this can be a commercial decision. The cost of swapping out is not prohibitive, especially if you plan from the outset to have NLU independence.

These tips will help you to deliver against the other key metric rarely discussed – that of enterprise chatbot agility – something that has proved particularly useful during the pandemic when customer advice was often changing by the day. For example, the Versus Arthritis chat service has needed to be updated on a regular basis over the last few months as clinicians provided the latest medical advice on the correlation between Covid-19 and Arthritis drugs.

This agility metric also applies to the continuous improvement and sophistication of your chatbot suite, most notably through integrations or infusing natural language technology and data science.

As an example of integrations, banking chatbots that can integrate with core banking systems will enable a far more elegant and personalised customer experience. But that requires a clean interface into those systems, and will not be a smooth transition if the chatbot domains have become unwieldy.

In terms of  infusing language and data sciences, it is possible to derive business rules from the customer text dialogue. Think raising alerts for adverse events or key words demonstrating customer churn risk, which can prioritise customer handling or escalation of the case.

In both of the above examples, the organisational design needs to mirror the technology architecture.

More to explore