In this blog, we’re sharing the story of our recent journey and the challenges to create a successful technical support process and build a strong support culture. A journey that started with a basic support system and gradually upgraded it to be more transparent and automated, fueled by a team of confident and skilled engineers.

Early days in support at Grabyo

Several years ago, our support process was always covered by a few key engineers who were able to help and assist our customers with any problem they could have. However, at some point, we started growing the team, and the current process was not scalable as it was not a good idea to depend only on a few engineers when we had a bigger pool of engineers who were capable of helping and eager to take on more responsibilities.

The initial rota was tracked in a spreadsheet, and it was as simple as checking which engineer was on support and calling that person.

The start of that journey

When we first proposed expanding our support rotation to our engineers, we observed some anxiety about providing support outside of office hours. Engineers assumed that we experienced many escalations during weekends and after regular hours, which caused them considerable apprehension. There were also concerns about their ability to resolve issues, given the complexity of specific areas on our platform.

We decided to collect data on platform issues from the previous years and generate metrics to gain better visibility of the current situation. By analyzing these metrics, we were able to identify the number of actual escalations and categorize them by teams.

Everyone was quite surprised and pleased to see that their assumptions were wrong and the number of escalated issues was lower than everyone thought. With this exercise, we were able to provide awareness of the current process and give more confidence to the new engineers being onboarded in the process.

What our Engineers thought of being in support BEFORE seeing support metrics.

What our engineers thought of being in support AFTER seeing support metrics.

To get a better understanding of our support process, we used several metrics, including the following:

• Escalations in the previous months based on priority
• Total escalations based on priority
• Escalations classified as P1 during office hours and out of office hours
• Escalations categorized by services or teams
• Escalations recorded by days

After making our data more visible and preparing everyone for what was coming, we began the onboarding process.

To give confidence to everyone who was part of the process, we started with multiple knowledge transfer sessions to train and refresh all the different areas that anyone in support could face. Each team focused on their products and services and provided multiple sessions where they explained what could go wrong and how to solve it. During this process, we also created some simple Runbooks to be used in case of issues.

We made sure to remind everyone of our priorities and aligned the first and second line support accordingly. Additionally, we defined the maximum amount of time that the engineering team should take to react to an escalated issue, which is known as the MMTA (Mean Time To Acknowledge).

🚫 P1	Urgent
P2	To be fixed in the short term
P3	To be fixed in the long term

During our analysis of the platform issues from the previous year, we discovered that some of the issues were false positives and others could be resolved by providing minimal training to the 1st line support team. To address this, we conducted multiple training sessions with the customer success team, who acted as 1st line support, to provide them with the necessary knowledge and tools to tackle any future issues.

Optimising the process

After onboarding new engineers and scaling the support rotation, the next step was to review, improve, and automate our existing process. We leveraged multiple tools to automate some of the necessary actions involved in this process.

To enhance this workflow, we began utilising tools that the team was already familiar with, such as Jira and Slack, and introduced new ones like Opsgenie and Airtable.

• Jira is used to monitor the platform issues, assign them to the appropriate team, and prioritise them.
• Slack is used to notify teams and establish specific channels related to escalated issues.
• Opsgenie is utilised to manage the support rota and automate the phone escalation process for the support engineer.
• Airtable is used to track metrics regarding all escalated platform issues.
• Intercom is the tool used by the Customer Success team to track all tickets and communications with our clients.

Here is more context of how our support structure consists:

The process consists of several components:

• The first line of support is responsible for handling all communication in Intercom and creating a platform issue in Jira.
• If the issue is classified as a P1, Jira will create a channel for it and notify the appropriate team in their support Slack channel.
• Jira will also send an alert to Opsgenie and escalate the issue calling the engineer who is currently on support duty.
• For P1 issues, use the dedicated Slack channel for communication. Use support channels for lower priorities.
• Airtable will sync with Jira to create and update all records for the dashboard.

Within this process, we also had to leverage Slack for communication and automate some steps to be able to filter the escalated issues and assign and notify them to the right team.

With all these changes in the support process, we have been able to improve the confidence that our engineers had about being on Support, but we have also improved the communication in handling the issues, and most importantly, we have increased the efficiency in solving and closing them.

Final Thoughts

Our journey to improve our support process has been marked by continuous improvement and strategic use of technology.

Empowering our engineers, fostering transparency, and embracing adaptability have been the key pillars of our success. As we continue evolving this process, measuring success with data, learning from failures, and prioritising employee feedback are essential. Moving forward, we are committed to delivering exceptional support, fostering a collaborative culture, and embracing automation in our ongoing pursuit of excellence.