Yapı Kredi Technology- Multimodal Next Generation Customer Communication Center Management System

NextConAI

Multimodal Next Generation Customer Communication Center Management System

It includes 3 partners from 2 companies within the scope of "Eureka Türkiye-Singapore Bilateral Call".

Project Objective

Improving Customer Experience with Video Call Analysis

It is aimed at improving service quality by analyzing the experiences of customers that use video call services.

Developing Subject-Focused Sentiment Analysis

It is aimed to obtain more in-depth insights by developing a new method that can analyze the moods of customers regarding specific subjects related to the Bank and its products.

Increasing Efficiency in Feedback Processes

It is aimed to achieve a more efficient feedback process by saving time and costs in the implementation of customer satisfaction surveys and increasing the participation rate in the surveys to 100%.

Improving Corporate Competence in the Field of Speech Processing

It is aimed to develop more effective solutions within the organization by enhancing the expertise in speech processing within Yapı Kredi Teknoloji.

Reducing External Dependency

It is aimed to reduce the need for external resources by developing speech processing and analysis applications within the organization.

Project Subject

It is aimed to measure customer satisfaction automatically, in real time and in depth by analyzing speech, text and image data through video calls with customer representatives.

Instant and Automated Satisfaction Measurement

The C-SAT score is automatically calculated based on sentiment and content analysis during the call, instead of traditional surveys, enabling real-time action.

Multimodal and Psychology-based Analysis

Multimodal data created by combining speech, text and images allows for detailed psychological analysis using sentiment models.

Customized Profile and Guidance

Interests are determined based on information obtained from customer calls, and promotions and recommendations are customized according to this profile.

What Do We Do?

Data Set and Labeling

Creating text and video data sets labeled with sentiments and aspects, increasing diversity with data augmentation techniques when necessary.
Supporting multimedia analysis projects by adding sentiment labels to video content.

Sentiment and Subject Analysis

Developing aspect-based sentiment analysis processes on texts in Turkish and English.
Designing systems capable of simultaneously predicting aspect and sentiment information using models such as Elmo, BERT, RoBERTa and DistilBERT.

Relation and Context Analysis

Applying dependency extraction, relation analysis and multi-task learning techniques to analyze the connection between aspect and sentiment.
Conducting analysis by considering past and future messages in order to improve context in dialogues.

Speech to Text Conversion

Compiling Turkish and multilingual open-source speech-text data sets to convert banking data into a processable format
Improving accuracy rates in speech-to-text conversion using language model-powered techniques such as LLM and KenLM.

Project Output

Multimodal and Real-time Customer Satisfaction Analysis System was Developed

An innovative artificial intelligence system measuring customer satisfaction through the integrated use of text, speech, image and financial data, and providing real-time sentiment and aspect analysis, was successfully deployed.

Accuracy and Performance Increase

An accuracy rate of 91% was achieved in speech-to-text conversion using the Wav2Vec2 model. A general accurate rate of 51% was achieved in aspect and sentiment analysis using the Sambalingo model. The success rate increased from 27% to 51% thanks to labeling quality and data diversity.

Advanced Modelling and Training Techniques

Model performance was improved using modern techniques such as instruction tuning, weighted loss and focal loss. Model routing capability was enhanced with task definition. Effective solutions were implemented to address data imbalances.

Implementation and Corporate Benefit

Analysis screens were developed specifically for representatives, and customer segmentation was enabled. Video labeling was completed to enable sentiment analysis from video calls. Technical know-how was gained in the field of in-house speech analysis.