A large national bank hired DecisivEdge to leverage natural language processing (NLP) to assign call reasons from unstructured text and introduce strategies that were previously not obtainable. The contact center solution also automated call topic labeling that previously relied on inaccurate/missing data supplied by call center agents.
Challenge
The project parameters defined by the client required the solution to be data driven, yet not model driven. Using a TF-IDF framework DecisivEdge met the project parameters provided by the client:
- Define categories of interest: Created a comprehensive list of hierarchical call reasons based on feedback from several corporate stakeholders.
- Build Dictionary/Lexical resource: Created a rule repository using significant word/phrases in a dictionary form to capture call reason.
- Create conditional logic: Some of the phrases and words define call reasons, others act as ancillary evidence and their presence could add more weight to the categorization logic.
- Validation: On classifying call reasons, manually assess classification performance.
Solution
The data analytics and contact center solutions teams at DecisivEdge used their NLP experience with contact center operations and financial domain knowledge to develop a solution on pre-existing call transcripts.
The contact center solution was a unique combination of natural language processing and heuristic logic, where holistic and meaningful text mapping is achieved through a powerful lexical resource in a dictionary form.
Below are three components that were used to define call reason logic and make sure the solution worked seamlessly:
- Dictionary: This plays a vital role in call reason capturing. Dictionary is created based on extensive study of call transcript for each call reason.
- Keyword identification: Using TF-IDF (a widely used method) to determine the importance of a word. This context was also used to promote a word to a keyword.
- Uni grams & Bi grams: For identifying phrases that would be part of defining a call reason.
All three components above were used to sync with business understanding and logic.
To understand the efficiency of the outcome, the following were addressed:
- False positives: Created primary and secondary reasons which would have 3 levels that summarized the ‘call reason’ efficiently.
- Lack of context: To include the importance of surrounding words, we used n-grams to determine the validity of any given match.
- Validation: Conducted manually on a randomly selected sample.
- Adaptability: Created an object-oriented rule bank that can be updated independent of the operational process.
Results
DecisivEdge’s contact center solution tested the performance of the new solution and confirmed that it outperformed the existing one.
- The new contact center solution had 63% and 67% accuracy for ‘in-time data’ and ‘out-of-time data’, respectively.
- The new solution had a ‘close match’ for call reasons with 25% and 21% accuracy for ‘in-time data’ and ‘out-of-time data’, respectively.
- The new solution had a combined 12% error-rate compared to the 30% audited error-rate with its current legacy system.
- The lift of the solution could be observed by understanding call reasons with ‘NA/NULL’. In the new solution – at least one reason is assigned to a call. However, the legacy process observes a ‘NA/NULL’ rate of about 27%.
In addition to the above results, DecisivEdge provided the client with a number of proof-of-concept strategies to further enhance the project’s ROI.
The new solution is currently in production mode and a Tableau dashboard is being developed for daily monitoring purposes.
*This page has been updated for accuracy as of June 2024.
