PORTFOLIO / INTERACTIVE
Virtual Consultant
for SKB Bank
Digital Embodiment of Natural Intelligence
PORTFOLIO / INTERACTIVE
Virtual Consultant
for SKB Bank
for SKB Bank
Client
SKB Bank
Task
Attract more visitors to the bank booth at the INNOPROM 2017 Expo and inform about new products and services.
Solution
Virtual Consultant as a Digital Avatar
RESULT
The project was a huge hit at the exhibition. People were lining up to ask a question to the virtual consultant. Some were interested only in banking products, while others mainly wondered about the personality of a charming digital lady. They all received comprehensive answers. Visitors were blown away by the level of knowledge and intelligence the virtual employee displayed as well as by her sense of humor and even self-respect. Artificial Intelligence experts were puzzled as for how it was even possible. Visitors' facial expressions changed from disbelief to either childlike excitement or fear in the face of new powerful technologies.
How We Did It?
In fact, it was the real human intelligence of the best bank employees that hid in the digital shell. Several actual operators in turns controlled a digital avatar. For that purpose we developed a unique hardware and software solution. It consisted of a visitor monitoring system, upper body and facial motion capture, and real-time 3D character animation and rendering software. Operator room had powerful graphic servers and was located 500 meters away from the booth. Therefore, we had to use a separate fiber line to avoid transmission delays of image and sound.
Milestones
3D Character Creation
Art concept, image, 3D modeling, texturing,
rigging, standby animation
rigging, standby animation
Motion Capture System Making
We made a special mocap helmet for facial motion capture. In addition, we used Vive Trackers to position the head, the upper body, and hands
Programming
We wrote an app with Unity. It uses body tracker positions and Faceware Software facial mocap data for real-time 3D character animation and rendering. There is also a standby mode in case when an operator is out or sensors are being calibrated
Booth Audio and Video Monitoring System Assembly
Thanks to the camera and the microphone over the 3D avatar, an operator could see and hear directly what booth visitors said
Voice Processing Setup
To make our character sound more 'digital', we used special software and a server to process an operator's voice on the fly
All System Installation at the Expo Site
We installed equipment at the booth and set up an operator's work place in a special secret room, located 500 meters away from the booth in another hall
Support during the Expo
Our team ensured continuous operation and system support throughout the 4 days of the exhibition