top of page

Chatbot improvements for BMO Assist

Background: introducing BMO Assist

In 2023, BMO's DMC team launched BMO Assist, a chatbot designed to support existing customers with quick answers to their queries.

Challenge: finding a matching response

Our fallback message was the default response (i.e., "I didn't understand that")  when the chatbot didn't understand what the customer was saying. 

It became a source of frustration for users because it occurred too frequently—about two-thirds of the time!

 

This made the chatbot seem less helpful and likely made customers avoid using it.

 

We had set a rule that only responses with a 60% or higher chance of being correct would appear.

 

But this often meant we missed chances to help users get the correct answer.

A poor user experience. The chatbot couldn't match a response even if it were close. 

Solution: present "close enough" responses

Initially, the product owner wanted to introduce slots to train our chatbot better. However, implementing it was lengthy and required a steep learning curve for the team. So, we went with the lowest-hanging fruit: improve the fallback experience. 
 

As an alternative to the repetitive "Sorry, I don't get it" response, we introduced a new approach called 'fall-forward.' The chatbot now presents customers with a few potential responses that are close enough matches (even below the 60% confidence score). 

old experience.png
old experience (2).png

So, instead of just saying, "Huh?" we suggest some answers. This could save time because maybe one of those suggestions is right.​

My role: designing UI-friendly CTAs for each response 

  1. Establish a UI-friendly content pattern for the fall-forward responses. 

  2. Developed 300+ fall-forward CTAs following the content pattern for consistency. 

  3. Create messaging for the fall-forward response. 

  4. Educate devs and QA on changes to the chatbot content with design documentation. 

Team

  • Developers​ (x2)

  • QAs (x3)

  • Business architect

  • Product owner

  • Agile delivery manager

  • Conversation designer (me) 

"Not sure if I got it right. Do you need information on any of these?

Locking a card

Cancelling a payment

None of the above

Content pattern for UI-friendly CTAs

Each one of these CTAs map to a response (i.e., intent). Use one of these formats: 

  1. [Verb ending in 'ing'] + [Noun]

  2. [Noun only] *only when the inquiry isn't about an action the user can perform (e.g., FAQ about bills)

  3. [Question]  *only if the other 2 patterns don’t work (e.g., Why am I receiving alerts?)

Max. character count: 48

Real examples

share_361692831610448279 (1).png
share_1993223009169839150 (1).png
share_5974216662990745890.png

Challenges

  1. Keeping the character count under 48 for French.

  2. Manually having to write and add responses to the chatbot spreadsheet and Amazon Lex. 

  3. Despite best efforts to make the CTAs all parallel with a verb-first content pattern, it was only sometimes possible. 

Outcome

Prior to the fall-forward implementation, the bot was able to find a response to a typed message about 45% of the time. After the fall-forward, we have almost doubled it at 86.5%!

13,150

Fall-forward responses invoked in the 1st week

90%

Increase in correct responses 

43%

 Responses contributed to total thumbs up

Get in Touch

Have any questions?

Fill out this form and I'll get in touch with you soon.

Thanks for submitting!

bottom of page