Welcome to the third and final piece in TheLoops AI in Action recap series. You can read part one of our AI in Action recap here. And part two here.
On December 4th, we hosted our inaugural “AI in Action” event where 9 CX leaders covered the ins and outs of AI implementation speaking on anything from driving employee adoption to selecting your first use case to measuring ROI.
For our final blog recap, we pick up with Kartik Yegneshwar’s session, “AI Alchemy: Transforming Complex Support Into Streamlined Solutions.”
Below, you’ll find a clip from the session as well as the full transcript of the conversation between Kartik and our CEO+Co-Founder, Somya Kapoor.
Kartik shares how he evaluated AI platforms and his team’s requirements by using the lifecycle of a ticket as a framework.
Somya Kapoor, TheLoops:
How are you doing?
Kartik Yegneshwar, Gainsight:
I’m doing all right. I actually ran upstairs before the last set of panelists came on to see if I should be wearing a blazer. And then I walked downstairs with it on and my wife kind of gave me the slow shake of her head. So, you know, I’m sticking to my sweatshirt for now rather than jumping into a blazer.
Somya Kapoor, TheLoops:
Oh, you know, Greg (Giletto) started that tradition and it went all downstream. This is really good. I’m enjoying all of the speakers listening, too. So, we started with Mohan talking about crawl, walk, run. And now we’re kind of coming back and getting to the closure of showing a different life cycle.
You and I have had discussions about B2B specifically, how you didn’t jump onto the chat bandwagon before everyone. You discussed bringing AI across the whole life cycle of viewing a ticket’s elements and really operationally driving that efficiency.
So, you know, for the leaders on our call today that want to understand your journey, and how you’re also coming from a background where you’ve tried multiple tools like Mohan and decided to go with TheLoops, can you share how started using AI within your environment and how you justified it at the management level as well?
Kartik Yegneshwar, Gainsight:
We started our journey about a year and a half ago. I did inherit another tool from my predecessor–I’ve been in my role for about two years now and we found that we didn’t think through our requirements really well in terms of what we wanted from an AI solution.
That led us down this path of looking at the life cycle of a ticket and, you in terms of how a ticket is really created.
- What happens before the customer creates a ticket for us?
- Do they explore self-service options?
- Do they look through our product documentation and so on and so forth?
And then, you know, what happens during the ticket creation?
- Once a ticket has been created and it’s active in the system, how is it routed to the right agent who can respond to that ticket in the first place?
- And what’s the customer sentiment?
- Is there some sort of agent assistance that’s available?
All of this speaks to improving the efficiency of our existing processes.
Support has been around at Gainsight forever, and there were existing processes on routing tickets in terms of helping our agents, do better and so forth.
What we were really looking from AI was to augment that process, in another words, augmented intelligence. That’s really what we were in the market for. And then finally, when the ticket gets closed out, what can we do with that data?
In the past, all we were doing was a CSAT survey and trying to understand how the customer was treated based on their responses.
We didn’t do any sort of QA simply because we just didn’t have the bandwidth to do it. It was something in the back of our mind, but we never got around to working on it. And some of the other panelists mentioned this, a lot of our data was stuck in our ticketing system.
It was not exposed to either other internal users outside of Support or our customers. So, what is it that we could do with that information? That’s something we wanted to explore as well.
And that’s why this framework of trying to figure out, ‘okay, where does the customer need help?’ is essential.
- Is it initially before the ticket is created, or is it, after the ticket is created and while it’s still open?
- Where we can drill down on the efficiency and give them a better experience
- Or is there something we want to do once the ticket is closed in terms of enhancing customer experience in the future?
So, we went down that path and that helped us identify what our requirements were from any AI system that we put into place.
Somya, you also mentioned chatbots. One of the things we realized when we drew out this picture and we realized it very quickly is the whole garbage in, garbage out still holds true, even with AI.
The success of a chatbot really depends on how good your systems are that actually provide you the information to give you responses. And we found we’re not there yet.
And a lot of demos that we saw in the market at that time, when we looked at a chatbot, the first demo that we always, “This is how you can change your password, or ‘This is your order status.’”You know, none of them related to us because in the B2B world, those were not the kind of tickets that we got. I mean, I wish I did, but no, we don’t.
The kind of questions we get, we don’t have structured data behind the scenes that a chat bot can easily pick up and respond to the customer.
Given the nature of that, where we felt we would get the most bang for our buck was really when the ticket is active and open because there were a lot of areas that we could hit from an efficiency standpoint and try to address those to give the customer a better experience while the ticket is open and active.
Then, later on, we can, go down the path of, once the ticket is closed out, what else can we do?
We were looking at CSAT prediction, but because again, I think the industry standard for any survey is about 10% response. I’ve got to be honest, I found it really hard to stand up in front of the board and say, ‘Hey, 10 percent of our customers really like us, and therefore our support is awesome.’
It’s a number I wasn’t very comfortable with. So we were definitely looking for a CSAT prediction engine. We were looking for other metrics that we could figure out at the end of tickets, when it’s closed out in order to better represent the quality of support that was provided to the customer.
Somya Kapoor, TheLoops:
Well, that’s really good. We’re getting a lot of thumbs up on this one, Kartik. You’re kind of addressing a very key component, that the kind of AI technology required for B2B is very different from B2C.
And yet in B2C, I do remember Monika sharing in one of our past conversations, if she ever had to do it again, she would do the backend optimization and AI for CX ops. Like you said, when the ticket is open for duration in B2B, operationalizing and optimizing efficiency on that than on the self-service side will give much more bang for the buck for the money.
So you nailed that element.
In this journey of figuring out the requirements, there’s two things that are happening:
- You want to buy a solution.
- You want to show to management the quick wins and the value associated with it.
There’s a lot to do with a platform like TheLoops and you’re on a journey of deploying everything. So, how did you pick and choose a quick win? How should support leaders on this call say, ‘Okay, let’s get one thing out the door so management sees the benefit while you have various other elements to optimize and operationalize from this cycle this year.’
Kartik Yegneshwar, Gainsight:
Yeah, it’s important to show that quick win so that can justify continued investment. And to mitigate any budgetary constraints that may come around.
I think one of the other panelists also mentioned, you know, you can’t flick a switch and make AI work.
It is a process. It is a journey that has to be drawn out.
So, when we started, we identified based on what we drew out in the picture, we said, “Okay, look, you know, efficiency is where we want to start and make our existing processes better. So, one of our big things was the intelligent routing.”
Before we had AI, one of the things we were doing was we had our support leaders essentially opening up every ticket, trying to identify what it’s about and sending it to the right agent. We at Gainsight have a large India team which works across shifts. We have a US team that works on one shift and then, we have a team in Europe as well that only works one shift.
The tickets can come in at any time predominantly during US business hours and EU business hours. So, how do you route it to the right agent?
Then there are different SLAs based on whether it’s paid support or standard support and you know all those things have to be taken into account.
We had our managers actually do that manually across all our tickets that were coming in and they would spend about 2-3 minutes per ticket in just not solving anything, but sending it to the right agent.
When we looked at this picture in front of us, that’s where we said, we can get max bang for our buck because given our support volume. We would easily save like one FTE count worth of hours in a month if we simply started with implement intelligent routing.
And even if the system would just give us like 80 % accuracy, that in and of itself is good enough for us to show a quick win.
That’s essentially why we chose that our as our first step.
And the fact that we can show those numbers fairly easily without doing much work in terms of trying to find the ROI, trying to find the efficiency and so on.
We already know these numbers. These were published numbers internally and putting intelligent routing into place with TheLoops actually doing the ticket classifications for us and then helping us find the right agent across geos with the right SLAs, keeping in mind the availability of the agents which was coming from a workforce management system. It was complex, but, that’s where we got the max bang for a buck.
Read more on the Gainsight team’s CX ops efficiency story here.
It was a pretty quick turnaround in terms of showcasing that win to my COO, which, helped us in our journey because now, internally, we have a lot of support for continuing forward with other areas that TheLoops is offering.
Somya Kapoor, TheLoops AI
That’s awesome, Kartik.
As you’re building it out, what would you pick as the next use cases in terms of your evolution?
How did you bring the team along in this process? Because to Mohan’s point, and Greg and Genady, some parts of the journey are pushback. How did you bring the team along?
Kartik Yegneshwar, Gainsight
To answer your first question, ‘how did I bring the team along?’
Just letting managers know that they wouldn’t have to do the routing manually was itself a no brainer at that point. The managers were on board.
Now for, for the agents, what we ended up working on was, once we implemented Copilot, it can be a long drawn out process where the feedback helps TheLoops understand what’s working, what’s not and make changes accordingly.
And one of the other panelists mentioned this before: it is a new muscle that needs to be trained.
What we did was go down the path of gamification where we had prizes and so forth, but like any program, it has its half-life period. It takes off and then drops off.
Where we are now is that we’re actually including the feedback to be provided to copilot as part of the metrics that we have our managers checking with their teams on a weekly basis.
So if you’ve closed out, let’s say 20 tickets, what we’re expecting is that at least for 80 percent of them, we want the agents to have provided some sort of feedback or use the ask feature.
So, that’s one thing that we’re kind of mandating as we move forward. But we did start off with the gamification and it did show us great success.
Before the gamification, we were like close to about 60% in terms of the likes and dislikes and the feedback provided. Then that quickly jumped up to like close to 75%- 78%. But then, we want to push it even further with kind of mandating that for now, because again, gamification doesn’t last forever.
Somya Kapoor, TheLoops AI
Do you now see your Copilot as the same as bringing on a new employee and jump starting them?
Kartik Yegneshwar, Gainsight
Yeah, that’s exactly the analogy that we’ve provided to the team in terms of, ‘Hey, you know, on the first day, we didn’t get all the right answers.’
It’s like training a new resource, you’re not going to get the best out of the resource on the first day, right? But as you train this resource, the resource just gets better and better over time. AI is no different.
Somya Kapoor, TheLoops
You’re basically think of it as training another human resource who’s going to be there with you forever and doesn’t forget things. And it is true, the team kind of loses the gamification, but you start seeing the benefits of the AI improving over a period of time.
Kartik Yegneshwar, Gainsight
Yeah, we’re currently measuring the success of Copilot as we speak. And the sentiment analysis has helped, we have implemented the impact scores within TheLoops which has helped us prioritize.
I think Lakshmi had spoken about this earlier. It helps us prioritize our tickets internally in terms of, ‘Which one should we attack first?’ and so on. And it helps the managers understand where they need to step in.
Also, the impact score keeps changing during the life cycle of the ticket itself, right? So, it also helps us understand, ‘hey, is something going well?’ Or, is something going downhill where someone more senior has to step in and help.
In addition to that, what we’re also doing is, you know the cost of ticket always increases as time goes by, right?
Because you’re moving from an L1 agent to an L2 to and maybe an engineering resource. Each of these resources costs more than the other.
So, one of the things we’re also trying to do is, I think Lakshmi alluded to this as well, is that we’re taking that impact score. And when it starts to get in the high or critical range where it happens to be a query that we have to run with engineering or a product defect that has to be open with engineering and they use JIRA, we are pushing that information into a place where engineering resources are able to understand that, “Oh, customer sentiment has gone from good to bad.”
Or that they’re asking for an ETA–which we really need to go back to them and respond to them on.
So it’s helping us downstream because one of the things that we didn’t do well earlier was, sometimes when customer sentiment goes poor, the Support team was aware of it, but the Engineering team wasn’t.
And now all that information is kind of flowing downstream.
And engineering is able to better prioritize their own tasks so that they help us out with the right tickets at the right time. Again, we, like every other company out there, we don’t have an unlimited supply of engineers. So we’ve got to be careful with how we use our resources.
Somya Kapoor, TheLoops
Oh, this is awesome. I love these stories and I love how everybody’s cross referencing each other. We’re going to create a community of support leaders like yourself that have been fundamentally transforming CX and bringing up things that were previously done manually. Do you see this helping you become a rockstar within your organization?
Kartik Yegneshwar, Gainsight:
I wouldn’t call it rockstar, but it definitely helps bring visibility into a lot of things that we’re doing from a support standpoint.
Sometimes, the efficiency gains are kind of lost in the noise because we in Support, we do understand that, ‘Hey, we routed a ticket faster and hence, the customer got help faster.’
But that may be lost in translation when we speak to folks outside of the Support organization because they may not appreciate how important that is.
One of the things that we’re working on is the the quality of support, right?
I had mentioned earlier that we were only relying on CSAT, and I was never comfortable with it. But now, we have the CSAT prediction engine in place. In addition to that, we have the customer attention score that we’re also using from TheLoops that helps us understand, you know, how much attention did we pay to this customer?
Ideally, what you want to see is for your high ARR customers, you want to see that extra attention being paid by our Support agents.
We’re starting to see that in the data and we can showcase that to our leadership to show that, ‘okay, we’re doing everything we can from a support perspective in order to help this customer as much as we can.’
From a quality standpoint, I would say CSAT prediction, the customer attention score overlaid on top of ARR and renewal dates comes really handy for CS teams and sales and renewal teams. But the other important aspect that we are in the middle of implementing right now is the AutoQA piece, right?
What that will help us do is us enable our agents better in terms of helping them with some key areas of the rubric that we’ve identified of our AutoQA process.
In addition to that, if we, slice and dice that information based on a customer basis, then it also helps us tell the customer, ‘Hey, look, you know, this is the kind of quality you’re receiving from a support standpoint, which is great because we have paid and standard support.’
So, it is yet another metric that we can share with our customers to showcase the quality that we bring to the table.
Somya Kapoor, TheLoops
This is awesome. Thanks, Kartik, and tying all that within the loop here, right?
With your journey cycle and everything bringing the benefits of various, not only just bringing efficiency within the organization, but having a better dialogue across organizations as well, using loops in the know how.
So maybe the last question for the leaders on the call, what are your final words of wisdom that you would say to someone that’s just getting started on this journey of getting AI deployed? What would you give them as a piece of advice?
Kartik Yegneshwar, Gainsight
Deploying AI is no different than deploying any other major system in your company.
- You first have to look at what are your requirements?
- Where are your quick wins?
- What’s the most bang for your buck you can get?
- And then, you know, find a partner who can partner with you. Along the journey.
When we started looking at TheLoops earlier this year, we wanted to focus more on the efficiency piece, and we found that you guys were really great at that portion.
You and I have had several conversations regarding chatbots. We’re not there yet. I know you have your own opinions about chatbots too, but the idea here is that we want you on this journey with us.
Also, when you pick a vendor, make sure you pick the vendor who has the strengths to address your primary requirements where you are going to start your crawl, walk, run journey, like it was mentioned earlier. You want to make sure that you’re partnering with someone who is really good at that crawl stage and can grow with you as you start walking and running. And, that’s one of the reasons we picked TheLoops.
Somya Kapoor, TheLoops
Thanks a lot, Kartik. It’s been a pleasure having you as a customer and this conversation.
To watch all of the sessions including Kartik’s, sign up for the replay of AI in Action today.
