Tomsk programmers "revived" the Dutch Santa Claus

TOMSK, Dec 15 – RIA Tomsk. The developers of the Tomsk company ENBISYS "revived" the Dutch Santa Claus – Sinterklaas, having developed a platform for interactive communication with him. How neural networks recognize children's desires and how the decision of the Tomsk citizens can change the global online education market – in the material of RIA Tomsk.

Same bot only different

"Do you believe in me or not?" – the Dutch Santa Claus – Sinterklaas is strictly looking from the laptop screen to the parent. "No", – he replies frankly. "Well, that's all, you have grown up, there is no point to talk to you". And falls asleep ... "Okay, okay, I believe", – the parent comes to his senses, he actually needs to try out the demo version of the platform and decide: whether it is worth to pay for the virtual meeting of his child with the fairy tale character?

Sinterklaas, opening his eyes and smiling, agrees to send a discount coupon to use the platform ...

"In Holland it is very expensive to call Sinterklaas to home, around 500 euros. And our partners with whom we worked on other projects offered to create a platform for virtual communication with a fantastic character", – the technical director of ENBISYS Maxim Usov tells background.

Simple applications for this could have been downloaded in the AppStore and PlayMarket before, but they mostly came down to chat, where at the other end of the "wire" there was a live operator at best, and most often – a bot. Tomsk citizens had the task not only to complement the video conversation, but also to make it interactive. In essence – to "revive" the wizard, to create the appearance of a conversation with a living character.

Let's goof on the grandfather?

"The technical challenge was to make it all work on modern browsers, on different platforms, so that it was not necessary to install anything. In development, there is the concept of "intrusive" (that is "annoying"), when a person needs to perform additional actions – for example, download and install the application. More than half will be cut off at this stage – just because of laziness", – says Maxim Usov.

Therefore, the interface have to be as simple as possible – to visit a website, click on the "demo version" button, try it yourself, and if you like it – pay for the child ... "And the simpler the interface looks, the harder to do it technically, because more automatic solutions are required , which means more work for programmers", – adds the ENBISYS programmer Stepan Krymsky.

The Dutch, for their part, filmed about a hundred video options with professional actors (Sinterklaas has assistants – Black Pitas, as well as an aunt), who then unnoticeably "stuck together" into a single script, depending on the child’s behavior.

"In the scenario there are about 10 points from which events may develop differently – depending on the child’s choice. For example, the wizard asks aunt to make him coffee, the camera focuses on her, she fills the mug, and then whispers to the child in a conspiratorial tone: "May be we put salt instead of sugar?" – gives the example Krymsky.

If a child answers "Yes!", the special algorithms distinguishing and analyzing the answer to the question, "transfer" the scenario to a plot, when Sinterklaas tastes coffee with salt and is indignant. If the child refused to play a trick on the grandfather, he just accepts a cup of coffee and quietly drinks it.

Virtual manager and teacher

Adults were slightly mistaken in how rough will be reaction to a meeting with grandfather: in promo videos, children sit grandly in front of the monitor and ask questions, in practice they enthusiastically shouted, saying "Sinterklaas! Sinterklaas!" through the word.

At the same time, for fast feedback, the system should clearly parse the child's speech. Google’s artificial intelligence built into the platform helps to isolate words from the audio stream.

"Google Cloud has very accurate voice recognition in the cloud on neural networks. We tested this system in Russian, letting it reproduce the phrase "sky-blue pink eye clapper with semi-broken legs". Recognized everything word for word! The net is trained on a huge number of examples, with different noises, speech defects, dialects, accents", – says Maxim Usov.

For the rapid delivery of voice from the browser to the cloud server and back is responsible the own technology of ENBISYS. Also Tomsk citizens have written algorithms which help recognize the meaning of speech and highlight standard behavioral patterns in order to instantly select the appropriate scenario depending on the child’s reaction.

"Solving this complex of tasks, we, of course, meant not just the opportunity to arrange a dialogue between the child and Sinterklaas. We could make a universal platform into which it is possible to download any material and work with it in real-time, based on the requests of a particular person", – emphasizes the director of ENBISYS Dmitry Bubnov.

This technology can be useful, for example, to banks. Now banking online services are reduced mainly to text chat, or, at most, voice assistants. "But no one offers a visual video along with this, and not just a talking head, but a real virtual manager. It’s difficult, and no one in the world did it yet", – says Bubnov.

Even more promising industry is – education.

"On the basis of our platform, together with TSU (Tomsk State University – Ed.), we are already making an adaptive learning system which will be fundamentally different from existing online courses. There are many units of educational content that can be glued together into an interactive lesson which will develop according to the student’s questions", – tells Dmitry Bubnov.

It will be the same individual learning – what is now practiced in universities to work with lagging students – but with a virtual teacher, that is, much cheaper and at the same time not less effective.