TWI783730B - Synchronize the "virtual audio and video at distributed sites" service system by using the central control entity personnel's actions - Google Patents

Abstract

The present invention provides a service system for synchronizing "virtual audio and video at distributed sites" by utilizing central control entity personnel's actions. Let customers get an excellent experience. After the front-end equipment captures the customer's audio and video, the glass information station displays the dynamic picture of the virtual character, and uses AI technology to automatically generate a synthetic human voice; Information modulation generates and modulates human voice output to the front-end equipment, and at the same time dynamically captures the behavior of service personnel to drive the avatar to move in a consistent manner on the dynamic screen. Among them, the synthesized human voice and the modulated human voice have the same timbre and audio range, and can seamlessly switch between AI and human services, enhancing the application strength of virtual-real integration technology.

Description

Synchronize the "virtual audio and video at distributed sites" service system by using central control entity personnel actions

The present invention is related to the field of customer service systems, especially a method of synchronizing multi-site virtual video and audio at the front desk with real people in the background through the concept of virtual and real integration, which can greatly reduce manpower allocation and realize undifferentiated switching experience. Synchronize the service system of "virtual audio and video in distributed sites".

Customer service, also known as customer service, is to provide customers with various required services. For a long time, customer service has been performed by human attendants. The current customer service can be roughly divided into "online customer service", that is, each customer connects to the customer service center by telephone, webpage and other remote methods, and then a real person connects to provide services by voice or text; or "online customer service" Point-based customer service”, that is, customers go to the physical counter to seek assistance from the service personnel stationed at each station. In order to provide instant customer service, companies generally need to deploy a considerable number of service personnel, especially for companies with distributed sites, it is necessary to ensure that there are enough manpower at each site. Inheriting this, there are also many previous technologies that combine related hardware devices to improve service efficiency, such as US20150059002, which uses virtual technology to develop hardware devices that can be worn by service personnel and customers and connected to each other. However, its application mode is only to solve the deficiencies of existing display devices in information display, and focuses on transforming physical written information or available information that can be quickly retrieved according to customer personal information conditions into a display that can be displayed to customers through hardware equipment. It is only a virtual screen for customers, and it is still a one-to-one service type in nature. The important thing is that you must wear the designated hardware when using it, which is still inconvenient in general. In addition, based on the uncertainty of customer demand, the allocation of service personnel is still It is difficult for various enterprises to handle the problem points properly. Two aspects of one body, when a large number of service personnel are set up, it will be able to respond to many customer needs, but the relative personnel costs will increase significantly, and some site personnel may be idle and service efficiency cannot be optimized; on the contrary, if personnel costs are considered and Reducing the number of service personnel may result in time-consuming waiting for customers, resulting in poor customer experience and negative impressions.

In view of this, the use of technology to improve the effectiveness of the aforementioned service areas should be a better consideration. With the development and progress of science and technology, people began to think about the possibility of machines having "intelligence", by imitating humans and thinking like humans. Subsequently, scholars from all walks of life invested in research one after another, and artificial intelligence (AI) technology was born accordingly. Different types and degrees of AI will gradually begin to replace some human operations, such as simple document processing, answering, or standardized manufacturing. Gradually, in the field of customer service, there are also voices to replace human resources with AI technology, thereby replacing human allocation and reducing personnel costs. However, although the current AI can achieve a very high response accuracy rate through training, its reply performance still does not break away from the cold appearance of traditional computer machines, for example, it can only reply with standard and emotionless words.

Later, in order to enhance the customer's operating experience, VR technology was combined with AI to enhance the sense of interaction and completely replace the service mode of real people. For example, patent documents such as CN111176442, CN112990043, and TW202121286 use VR and AI intelligent technology. Let the user enter the virtual environment to receive guidance by wearing a monitor, and then use AI technology to recognize the user's instructions based on the VR environment, and then execute the required services. Although the above-mentioned technology further utilizes virtual technology to enhance visual effects, it still cannot get rid of the cold feeling of AI standard response and mechanical action. At the same time, in the above-mentioned technical teaching, users must wear corresponding equipment before they can use it. Convenience and cleanliness are still not perfect. In addition, these technical documents are established based on the concept of special membership attributes, so they can only be applied to, for example, the financial field listed in the documents or other members with membership It can be effectively applied to the enterprise field, not to any user. Therefore, even if virtual customer service is combined with VR technology, it still cannot give customers warm interactions like real people, and there are many inconveniences in actual use. This is the flaw of traditional virtual customer service.

Based on this, real people and AI technology are at both ends of the balance, whether it is to completely replace real people with AI, or continue to maintain the status quo of real-person services, choosing either side has its insurmountable shortcomings. Accordingly, there are also service system technologies that intend to combine AI and real people. For example, TW202030626, TW202032392 and other patent documents. However, although these technologies overcome the lack of using only real people or only using AI technology, their service provision is still dominated by AI, and only AI can determine all processing orders, and real people can only act as assistants called by AI , In other words, these technologies do not substantially integrate AI technology and real-life services, at best, they only coexist with separate actions of the two, and then switch between AI services and real-life services as appropriate, for example, completely using pure AI services Convert to a pure human service, or completely convert from a pure human service to a pure AI service. In other words, the AI and human services in these technologies are all single-tasking operations, and there is no application state where AI and real people work on the same customer at the same time. In addition, because this technology is applied in the financial field and has high privacy requirements, in fact, these documents focus on how to properly hide user information and then perform intelligent judgment processing technology, rather than improving customer interaction. purpose, and is not an open customer service system that anyone can use. These solutions that rely heavily on AI as a virtual customer service can achieve the effect of reducing labor costs, but in fact, it is impossible to detect improper handling or errors in AI in real time, which can easily lead to emergencies when users are already unhappy Switching to real-person service will cause customers to feel bad, and in the part of switching, it will produce a quite obvious sense of abruptness to customers. In terms of user experience, this point is still not considered to be improved and solved by the existing technology place. And as the "online customer service" proposed by CN112995568, it is only in the traditional online webpage customer service The front-end AI is slightly introduced into the system to help quickly classify merchants and establish technical teachings for each merchant's chat group. This part has nothing to do with the key points of the development of consulting applications and the elements that need to be considered when customers need to visit the cabinet.

In view of the many deficiencies of today's "site-based customer service" and the remaining defects of the above-mentioned technical documents, in order to achieve a configuration balance between real-person services and virtual intelligence, the team of the present invention has integrated virtual and real interactive technologies for many years. Explore and research, further conceive and propose a service system that absorbs the advantages of AI technology and physical personnel and can use high-speed networks to meet the real-time needs of a large amount of information synchronization. Synchronized control can not only eliminate the cold feeling of AI technology, but also greatly reduce the number of physical staffing, and allow customers to have a warm interactive experience with virtual images, connecting the real and virtual worlds.

One purpose of the present invention is to provide a service system that utilizes the actions of central control entity personnel to synchronize the "virtual audio and video of distributed sites". The service provides the screen, and the screen is built on a transparent display to make the type of the site more variable, and the background control personnel can dynamically control the virtual audio-visual performance synchronously, thereby realizing one-to-many service performance, and in the Realize seamless switching between real people and AI to provide customers with a more humanized virtual screen interaction experience.

In order to achieve the above purpose, the present invention proposes a service system that utilizes central control entity personnel to synchronize the "virtual audio and video of distributed sites", including: a plurality of front-end equipment, including: a front-end host with an audio-visual recording device , for extracting multiple voice information of at least one customer, including: an image generator for generating a virtual character information; a semantic analyzer for receiving the voice information, and using an artificial intelligence engine locally or in the cloud calculating and judging the voice information to form an automatic response information; and a human voice simulator connected to the semantic analyzer by telecommunication for receiving the automatic response information and Generate a synthetic human voice; a glass information platform, part or all of which is a transparent display, connected to the image generator by telecommunication, and receive the information of the virtual character to display a dynamic picture accordingly; and a loudspeaker, connected to the human voice simulator by telecommunication, To receive the synthesized human voice output; a background device, connected to the front-end devices through Internet telecommunications, for at least one service personnel to operate on the real-time audio and video of the customer captured by the audio-visual recording device, including: at least one The central control server is connected to the front host by telecommunication, and has an authority switcher, and the authority switcher is telecommunication connected to the loudspeaker; at least one voice-tuning microphone is connected to the central control server by telecommunication, for receiving the service personnel Input a manual response information, and generate a modulated human voice according to the manual response information modulation for the speaker output; at least one audio-visual playback device, connected to the central control server by telecommunications, for displaying the audio-visual recording device captured The real-time audio and video of the customer, and the dynamic picture; and at least one motion capture device, telecommunications connected to the central control server and the audio and video playback device, to capture the behavior of the service personnel, and use the behavior to drive The virtual character information is a consistent action; wherein, the number of the front-end devices is N, the number of the voice-tuned microphones is M, and N>M; in addition, the authority switcher automatically or manually switches the speaker to receive the synthesized human voice or The modulation vocal authority, and the synthesized vocal and the modulated vocal have the same timbre and the same audio frequency range. According to this, customers can use any of the front-end equipment to interact with the dynamic screen of the avatar, and with the back-end equipment in the central control server, the service personnel can immediately pay attention to the customer situation of each distributed site and operate it. Realize the excellent service efficiency of one person serving multiple stations; at the same time, the behavior of the service personnel can be consistent with the information of the virtual character, so that customers can have a more friendly and humanized interactive experience.

Further, in order to improve the reliability of service provision and customize the performance of services required by different customers, the audio-visual playback device has a touch screen, and the touch screen is used by the service personnel to input at least one gesture control, and Accordingly, the dynamic picture is executed synchronously.

Based on the content of any of the foregoing embodiments, wherein the image generator further generates information information, and the glass information platform simultaneously displays the dynamic picture containing the information of the virtual character and the information information after receiving the information of the virtual character and the information information , so that information presentation can be expanded and a more complete service description can be provided.

In addition, in order to prevent the artificial intelligence engine from being unable to provide the services required by customers smoothly, resulting in long-awaited and unpleasant user experience, when the content of the automatic response information repeats beyond a threshold, the authority switcher is automatically switched The speaker receives the modulated vocal output for instant response by the attendant.

In addition, in order to improve the applicability of the service system, the front host of the present invention further includes: a language recognizer connected to the semantic analyzer by telecommunication for judging the language of the voice information, so that customers can be quickly identified The language used, and then express and reply in the corresponding language. For customers in non-native-speaking countries, this can greatly reduce the difficulty of service consultation and allow customers to have more excellent and considerate use experience.

Based on the foregoing embodiments, further, the language recognizer stores a defined language-recognized word for the customer to use the language-recognized word as a pronunciation expression, and uses the sound information as a basis for language-recognized judgment, whereby, It helps to speed up the identification of the customer's customary language and reduces the probability of identification errors.

Based on the content of the above two embodiments, the central control server further includes: a language matching device, which is connected to the language recognizer by telecommunications, and after judging the language of the voice information, automatically matches to the corresponding voice-tuned utterance When the language recognizer successfully identifies the customary language of the customer, it can be matched to the voice-tuned microphone of the corresponding language, so that the service personnel with the ability to communicate in the language can serve it.

In order to quickly and accurately grasp the service usage status of each distributed site and the average workload of each service personnel, the central control server further includes: a point marker and an automatic matching device, wherein, when the voice tuning When the number of microphones is more than two, the automatic matching device is connected to the voice-tuned microphones to distribute the working times of the voice-tuned microphones evenly, and the location marker shows the location of the front-end equipment, and the The connection status between the foreground equipment and the voice-tuning microphones.

Based on the aforementioned embodiment, further, the central control server further includes: a manual matching device, which connects the voice-tuning microphones by telecommunications and gives priority to the automatic matching device to force work matching, wherein the automatic matching device is based on a The preset value is automatically allocated, and when the voice-tuned microphones reach the preset value, the automatic allocation is excluded. According to this, the system can solve some emergencies under automatic allocation, so that each service personnel can support each other to improve the overall service situation, and after each voice-tuning microphone has reached its working frequency, each service personnel can control it by themselves Subsequent service distribution.

Based on the present invention, it is possible to switch between artificial intelligence and real people. In order to make customers more insensitive to the switching behavior and to allow service personnel to quickly grasp the current service progress, the central control server further includes: at least one text compiler, telecommunications connection The semantic analyzer is used to sequentially compile the voice information and response information for the customer into text information and display it through the audio-visual playback device.

To sum up, the service system of the present invention, which utilizes the central control physical personnel to synchronize the "virtual audio and video at distributed sites", is to provide all customers with warmth and emotion even when communicating with virtual characters on dynamic screens. The system architecture conceived and planned based on the humanized interactive experience as the starting point. The present invention has many advantages in various aspects. For the enterprise aspect, the customer service resource allocation of the distributed site, the present invention uses less than the number of front-end equipment to tune the microphone, and the virtual character dynamic picture on each front-end equipment This method realizes the effect that one service person can serve multiple site customers. From the customer's point of view, the present invention uses the synthesized human voice and the modified human voice with the same timbre and the same audio range Voice, to achieve the effect of seamless switching between real people and artificial intelligence, so that when customers use the front-end equipment services, they will not feel abrupt due to the switching of service objects, but have a consistent experience of use. As far as service personnel are concerned, the real-time audio and video of customers captured by the front-end equipment and the audio-visual playback device in the background can better and comprehensively respond to the status of each customer and provide the required service. Furthermore, based on the perfection of the overall system in all aspects, the present invention also proposes more detailed technical features, as described in the above embodiments.

1: Foreground equipment

10: Foreground host

101: Audio-visual recording device

102: Image Generator

103:Semantic Analyzer

104: Vocal Simulator

105:Language Identifier

11: Glass Information Desk

12:Speaker

2: background equipment

20: Central control server

201: permission switcher

202: Text Compiler

203:Language matcher

204: Site marker

205: Automatic Matcher

206: Manual Matcher

21: Tuned microphone

22: Audio-visual playback device

221:Touch screen

23:Motion Capture

8: Service staff

9: Customer

A: Internet

D: dynamic picture

Figure 1 is a schematic diagram of the service system architecture of the preferred embodiment of the present invention.

Figure 2 is a schematic block diagram of a service system module in a preferred embodiment of the present invention.

Figure 3 is a schematic diagram (1) of the application of the service system of the preferred embodiment of the present invention.

Figure 4 is a schematic diagram (2) of the application of the service system of the preferred embodiment of the present invention.

Figure 5 is a schematic diagram of the application of the service system of another preferred embodiment of the present invention.

The key point of the present invention is to construct the system of "entity site customer service", which is quite different from the traditional online customer service in essence, and does not have an equal and comparable technical basis, which will be described first. As mentioned above, in the current field of customer service, whether it is purely human customer service or AI intelligent customer service, each has its own shortcomings, and it is impossible to completely replace the other with one model. Even if there is a service technology that combines the two, it still only allows real people to play the supporting role when AI cannot handle it, and puts too much emphasis on AI intelligent processing. Wearing various hardware devices can cause a lot of inconvenience. Therefore, in order to provide a better customer service system, effectively eliminate the cold feeling of intelligent machines so that customers With a warmer and more humane interactive experience, the invention team is based on years of experience in developing and researching virtual technology, effectively applying AI intelligent technology and virtual technology to "entity site customer service", and does not need to be stationed at the station Real-person waiters at each point assist and cooperate with each other to build a brand-new service system that essentially integrates AI and real-person services, so as to synchronize the "virtual audio and video of distributed sites" by using hollow physical personnel movements, in addition to maximizing the efficiency of real-person services , It can also give all customers a good experience.

In order to enable those skilled in the art to clearly understand the content of the present invention, the following descriptions are provided together with the drawings for your reference. Please refer to Figures 1 to 5, which are a schematic diagram of a service system architecture, a schematic diagram of a service system module block, a schematic diagram of a service system application (1), a schematic diagram of a service system application (2) and another preferred embodiment of the present invention. Schematic diagram of the application of the service system of the embodiment. The service system of the present invention for synchronizing the "virtual audio and video of distributed sites" by using the actions of central control entity personnel includes a plurality of foreground devices 1 and a background device 2 .

The foreground equipment 1 includes a foreground host 10 , a glass information stand 11 and a loudspeaker 12 respectively. The front host 10 has an audio-visual recording device 101 for capturing multiple voice information of at least one customer 9 , and includes an image generator 102 , a semantic analyzer 103 and a human voice simulator 104 . The image generator 102 is used to generate a virtual character information, and the semantic analyzer 103 receives the voice information, and calculates and judges the voice information through a local or cloud artificial intelligence engine to form an automatic response information. The human voice simulator 104 is connected to the semantic analyzer 103 by telecommunication for receiving the automatic response information and generating a synthetic human voice. Part or all of the glass information stand 11 is a transparent display, which is connected to the image generator 102 by telecommunication, and receives the virtual character information to display a dynamic picture D accordingly. The speaker 12 is connected to the vocal simulator 104 by telecommunication for receiving the synthesized vocal output. Specifically, the glass information platform 11 refers to a fixed or movable information providing device, such as used in the field of public transportation When used, the glass information desk can be a service desk in each station or a non-embedded large-scale notice board, or it can be a glass window of a train, an MRT or other means of transportation; when applied to various enterprise fields, it can be The entrance counter of the building; in the automotive field, it can be the glass window of the car. Moreover, the glass information station 11 specially selects a transparent display, which can achieve the effect of maintaining the perspective of the environment in the non-use state, and based on this, the distributed site can be presented in the form of car windows, glass windows, etc. , instead of having a more abrupt feeling in the environment like a simple interactive host or a traditional display.

And this front host 10 can be the hardware device that is arranged on this glass information platform 11 in principle, or is the device that combines hardware and cloud platform to form, when these elements and its information computation utilize cloud to realize, system can reach more Good computing processing efficiency, speed and resource utilization can also improve the immediacy of information transmission. And this loudspeaker 12 then is arranged according to actual application situation, when for example this glass information platform 11 is service desk, then can be directly arranged on this glass information platform 11, if this glass information platform 11 is the vehicle window of vehicle, then can It is arranged in a position in the car body that is beneficial for customers to listen to, or directly arranged in the area where the glass information desk 11 is a transparent display.

The background equipment 2 is connected to the front equipment 1 through the Internet A telecommunications, and at least one service personnel 8 is used to operate the real-time audio and video of the customer 9 captured by the video recording device 101. The background equipment 2 includes at least one A central control server 20 , at least one voice-tuning microphone 21 , at least one audio-visual playback device 22 and at least one motion capture device 23 . The central control server 20 is connected to the front host 10 by telecommunication, and has an authority switcher 201 , and the authority switcher 201 is connected to the speaker 12 by telecommunication. Similarly, the central control server 20 can also be a hardware device, or a virtual group built on the cloud. The voice-tuned microphone 21 is connected to the central control server 20 by telecommunication for receiving a manual response information input by the service personnel 8, and generates a modulated human voice according to the manual response information for output by the speaker 12. The audio-visual playback device 22 is telecommunications connected to the central control server 20 for displaying the real-time video of the customer 9 captured by the audio-visual recording device 101 sound, and the dynamic picture D. The motion capture device 23 is telecommunications-connected to the central control server 20 and the audio-visual playing device 22 for capturing a behavior of the service personnel 8, and using the behavior to drive the virtual character information into consistent actions. Wherein, the number of the foreground devices 1 is N, the number of the voice-tuning microphones 21 is M, and N>M; the other authority switcher 201 automatically or manually switches the authority of the speaker 12 to receive the synthesized human voice or the modulated human voice , and the synthesized vocal and the modulated vocal have the same timbre and the same audio frequency range.

In the present invention, the quantity M of the voice-tuning microphones 21 in the background equipment 2 is less than the limit condition of the number N of the front equipment 1, so that the present invention can realize the service efficiency of one person to multiple sites, effectively reducing Labor cost, at the same time, the synthesized human voice and the modulated human voice have the same timbre and audio frequency range, so that the customer 9 will not have a sudden sense of conversion, and realize the seamless conversion between real people and intelligent machines, so that the customer 9 Have a better experience. Going back to the background device 2, specifically, under such a framework, no matter how many front-end devices 1 the service system has, they can all be monitored and correspondingly operated through a single background device 2 . And this central control server 20 can be built on the cloud platform based on the current cloud technology and high-speed transmission network environment. Catcher 23 etc. are arranged in the place where this service personnel is. Through the background equipment 2, the service personnel 8 can monitor the status of each front equipment 1 at any time, and the audio-visual playback device 22 can display the real-time audio and video of the customer 9 and the dynamic picture D of the glass information platform 11, so that the The service personnel 8 can intervene at any time and input the manual response information through the voice-tuned microphone 21 to convert it into the modulated human voice, and then output it to the customer 9 through the loudspeaker 12 . And through the dynamic capture device 23, the service personnel 8 can respond to the status of the customer 9 at any time, so that the virtual character information is consistent. In other words, the actions of the service personnel 8 can be synchronized and real-time through the dynamic screen D is displayed to the customer 9, for example Such as specifying a certain piece of information with an action, etc. Therefore, through the present invention, it is possible to achieve rapid switching between real people and artificial intelligence, as well as synchronous operation of dynamic images of virtual characters, thereby greatly improving service quality.

Generally speaking, the service system of the present invention enables the personnel directly facing the customer 9 to present in a virtual way, giving people an eye-catching and refreshing user experience, and the customer 9 can directly pass through the glass without wearing any device The information station 11 watches the dynamic picture D and interacts with it to inquire. Accordingly, it is no longer necessary to have a certain space at the site due to the need to configure real people in each distributed site as in the past, so that the application of the glass information station 11 can be further achieved. Extensive expansion possibilities. In the process of use, in addition to being able to respond to the customer 9 through artificial intelligence engine calculations, the service personnel 8 in the background can also monitor the status of each front-end device 1 at any time, and use the authority switcher 201 to allow the customer to The service personnel 8 intervene to take over the service, which can greatly reduce the probability that the customer 9 will be unhappy because the artificial intelligence is not enough to assist. And the response of the service personnel 8, if it is voice, will be output with the modulated human voice with the same timbre and audio range as the synthesized human voice, and the body movement part will make the virtual character information and the service through the motion capture device 23 The actions of the personnel 8 are consistent, and the instructions are strengthened through gestures, so that the customer 9 can have the warmth of interacting with a real person even though he sees the image of a virtual character, and enhances the humanity displayed by the virtual character. In particular, the design basis of the service system of the present invention is based on the high-speed Internet A to realize real-time information processing and image transmission. The present invention is effective according to the precondition of implementing. Therefore, in terms of information processing, transmission, and calculation requirements, the present invention is in a completely different concept and application field from the past customer service systems that simply use telephone voice, web text applications, and the like. Since the previous service system platform did not exist in the high-speed network environment in this space-time background, even if the conventional service system platform further uses AI or VR technology, its technology can only be developed and established based on the existing network speed specifications at that time basis, and jointly, its information transmission and The functions are bound to be limited, and it is impossible to realize the information processing performance as in the present invention and the system technology that can be achieved based on the complementary construction of real people and AI.

It should be noted that the focus of this invention is to draw on the advantages of artificial intelligence and real customer service to establish a more complete service system. Therefore, the analysis or analysis capabilities of artificial intelligence technology and the fineness of virtual screen performance are all incomparable. The focus of the present invention, as mentioned above, is the new service system scenario that combines artificial intelligence and real people into virtual services. This is the focus of the present invention. Therefore, the overall structure of the present invention should not be disassembled arbitrarily.

Further, in order to improve the service quality of the service personnel 8, the audio-visual playback device 22 has a touch screen 221, and the touch screen 221 is for the service personnel 8 to input at least one gesture control, and accordingly the dynamic picture D Actions are then executed synchronously. For example, for an older customer 9, the service personnel 8 can amplify or move specific information synchronously when explaining, so that the customer 9 can be more easily understood.

In order to make it easier for the customer 9 to know the required service information quickly, the image generator 102 further generates information information. After the glass information platform 11 receives the information of the virtual character and the information information, it simultaneously displays the information including the virtual character. Information and the dynamic screen D of the intelligence information. For example, when the customer 9 asks about the ticket price, the dynamic picture D can be made to include the intelligence information of the fare, or when the customer 9 asks about the location of the restroom, the dynamic picture D can also be made to include the area The intelligence information of the floor plan. At this time, if the audio-visual playback device 22 has the touch screen 221, the service personnel 8 can input the gesture control through the touch screen 221 to enlarge the fare information or the toilet location for the customer 9 to watch.

As mentioned above, the authority switcher 201 of the central control server 20 can automatically or manually switch the loudspeaker 12 to receive the synthetic voice or the modulation voice authority, and the automatic switching of the authority switcher 201 is based on a comparison In the best implementation mode, when the content of the automatic response message repeats more than one When the threshold is reached, the authority switcher 201 automatically switches the speaker 12 to receive the modulated human voice output. For example, if the automatic response information is repeated more than 3 times, it will be considered that the situation is that artificial intelligence cannot judge the content represented by the voice information of the customer 9, and then cannot give an appropriate reply. At this time, the authority switcher 201 automatically switches The loudspeaker 12 receives the output of the modulated human voice, allowing the service personnel 8 to intervene in the service provision, which can prevent the customer 9 from losing patience or feeling unhappy when receiving repeated responses, effectively and quickly solving the situation that artificial intelligence cannot handle . In fact, there are many ways to judge whether artificial intelligence cannot give the customer 9 appropriate services. However, in view of the speed and efficiency of information discrimination, avoiding complicated program operation does not necessarily have better accuracy. In the embodiment, it is specifically proposed that the content of the automatic response information repeatedly exceeds the threshold as the judgment criterion, which not only improves the judgment efficiency, but also has a more intuitive humanized judgment performance.

Further, in order to facilitate the service personnel 8 to better understand the interaction status between the customer 9 and the front-end equipment 1, such as inquiry content, etc., in an implementation, the central control server 20 further includes at least one text compiler 202 , connected to the semantic analyzer 103 by telecommunications, for the voice information and response information directed at the customer 9 to be sequentially compiled into text information and displayed through the audio-visual playback device 22 . In this way, in addition to watching the real-time audio and video of the customer 9, the service personnel 8 can also understand the exact content of the voice information more clearly, especially when undertaking the service work of the customer 9 that is first answered by artificial intelligence in the previous stage, text The information system can greatly help the service personnel 8 understand the cause and effect, so as to seamlessly continue the service, so that the customer 9 does not need to repeat his needs, and this can also make the customer 9 less sensitive to the conversion between real people and artificial intelligence .

The service system of the present invention can be applied in various fields, and it is also possible to face the situation of providing services for the customers 9 of various nationalities. In order to serve the customers 9 reliably in this application situation, in a In the implementation mode, the front host 10 further includes a language recognizer 105, which is connected to the language by telecommunications. The meaning analyzer 103 is used for judging the language of the audio information. In this way, the front host 10 can automatically analyze and judge the language type, and further form the synthetic human voice corresponding to the language, so that the customer 9 can not use it smoothly because of the language type problem. This point is especially important when the service system is applied to places with more foreign tourists such as airports and stations.

Further, in order to improve the recognition speed of the language by the front host 10, and to make the customer 9 more conducive to expressing the type of language used by the customer 9 with a simple vocabulary, the language recognizer 105 can store a defined language recognition Recognized words are provided for the customer 9 to use the recognized words in the language as pronunciation expressions, and use the voice information as a basis for language judgment. In this way, the language recognizer 105 can greatly reduce the amount of the sound information that needs to be judged, and only use at least one of the sound information for the recognized word of the language to quickly identify. The language identification word can be, for example, a continuous number, such as "123", or a simple greeting, such as "Nihao".

After the front-end equipment 1 can identify the language of the customer 9, the background equipment 2 can be assigned accordingly to the service personnel 8 who have the corresponding language ability to respond and explain when needed. Specifically, the central control server The device 20 further includes a language matcher 203, which is connected to the language recognizer 105 by telecommunications, and automatically matches to the corresponding voice-tuned microphone 21 after judging the language of the voice information. In this way, the service personnel 8 who have the language-specific ability can be assigned to the customer 9's customary language reliably and automatically, and respond and interact in the language type, giving the customer 9 a more intimate experience and improving service quality.

And at the background equipment 2 end, in order to facilitate grasping the use status and assignment work of each of the front equipment 1, in an implementation mode, the central control server 20 further includes a point marker 204 and an automatic matcher 205, Wherein, when the number of the voice-tuning microphones 21 is more than two, the automatic matching device 205 is connected to the voice-tuning microphones 21 and equally distributes the working times of the voice-tuning microphones 21, and the position is marked The device 204 displays the positions of the foreground devices 1 and the connection status of the foreground devices 1 and the voice-tuned microphones 21 . Specifically, the position indicator 204 can display or use physical LED lights to indicate the position of each front-end device 1 and the connection status with the voice-tuned microphones 21 . For example, the glass information desk 11 of each front-end device 1 can be named, and the voice-tuning microphone 21 etc. currently connected to the front-end device 1 can be displayed in the same light color mode, or can be directly displayed through the display. This foreground equipment 1 name and this voice-tuning microphone 21 serial numbers etc. all can be connected with it. The automatic matching device 205 is used for evenly distributing work. In principle, one service personnel 8 corresponds to one voice-tuning microphone 21. When the background equipment 2 has more than two voice-tuning microphones 21 , in order to make each of the service personnel 8 have the same workload, the automatic matching device 205 will evenly distribute the number of times that each of the voice-tuning microphones 21 should be performed, so as to achieve the purpose of fair operation, and thus avoid service Work is too concentrated on one or a few service personnel.

In actual application, there may be many emergencies that need to be eliminated in time by the service personnel 8. Therefore, further, the central control server 20 further includes a manual matching device 206, which is connected to the voice-tuning microphones 21 by telecommunication and prioritized. The automatic matcher 205 is forced to perform work matching, wherein the automatic matcher 205 performs automatic allocation according to a preset value, and when the voice-tuned microphones 21 each reach the preset value, the automatic allocation is excluded. For example, when the automatic matching device 205 assigns work to any one of the voice-tuning microphones 21, but the corresponding service personnel 8 happens to be absent due to business, the remaining service personnel 8 can be assigned by the manual matching device 206. And transfer the service work to the tone-tuning microphone 21 of itself, to assist in completing the service work in real time. Or the service personnel 8 assigned by the automatic matching device 205 may not be able to effectively solve the problem of the customer 9. At this time, the rest of the service personnel can also be transferred to the voice-tuned speech of themselves through the manual matching device 206. Device 21 to assist, so that the effect of allowing each of the service personnel 8 to support each other at any time can be achieved.

And about the distribution of the number of times of work of the automatic matching device 205, in addition to constantly evenly distributing, the preset value can also be further set, so that after the number of times of work of each of the voice-tuning microphones 21 reaches the preset value, it can be changed to The manual mode allows each of the service personnel 8 to undertake the service work of assisting the customer 9 by himself. The preset value can be set according to each application state. For example, when the glass information desk 11 is a service desk in a station, the preset value can be calculated and set according to the total number of stations and the number of users, that is, Each of the service personnel should reach the work threshold, so that the work distribution can be more equal, and the uneven workload caused by human factors can be eliminated. After reaching the work threshold, the work of these voice-tuning microphones 21 is not mandatory, but each service personnel 8 is allowed to decide on their own, so that an additional system can be formed to encourage these service personnel in personnel management.

The following is an example where the glass information desk 11 is set up in a station as a platform to provide related services, and briefly illustrates various application scenarios. As shown in Figures 3 and 4, the transparent display of the glass information platform 11 displays the dynamic picture D with the virtual character information and the intelligence information. When the customer 9 approaches and starts to speak, the audio-visual recording device 101 The voice information of the customer 9 is captured, and the real-time audio-visual transmission is displayed on the audio-visual display device 22 in the background device 2 . At this moment, the semantic analyzer 103 of the foreground device 1 will start to judge the sound information through the artificial intelligence engine calculation to form the automatic response information, and then generate the synthetic human voice by the human voice simulator 104 and output it from the loudspeaker 12, Then respond to the needs of the customer 9, as shown in Figure 3. During this process, the service personnel 8 can switch and monitor the operation status of each front-end device 1 at any time through the audio-visual playback device 22, and can intervene for support at any time. For example, if the service personnel 8 finds that any of the customers 9 are expressing doubts, then the service personnel 8 can move their hands to indicate a certain direction, and the behavior is captured by the motion capture device 23 and immediately drives The avatar information moves in unison. At this time, the avatar information of the dynamic picture D seen by the customer 9 will move synchronously in real time to indicate a certain direction, as shown in Figure 4. Show. Alternatively, when the audio-visual playing device 22 has the touch screen 221, the service personnel 8 can also enlarge or mark a certain information block through gesture control, so that the customer 9 can understand it more easily. And when many customers 9 use the front desk device 1 respectively, the authority switcher 201 can automatically switch the loudspeaker 12 to receive the modulated human voice output when the number of repetitions exceeds the threshold, so that real people can take over the service.

Another usage scenario is that when there are many customers 9 using different front-end equipment 1 at the same time, if there are many customers 9 who all need the assistance of the service personnel 8, they can be automatically activated through the authority switcher 201. Switch to each of the service personnel 8 to take over and utilize the voice-tuning microphone 21 to reply to the customer. Or when the automatic matching device 205 is provided, the service operation of each of the customers 9 can be evenly distributed to each of the voice-tuning microphones 21 by it. If the voice-tuned microphone 21 assigned to the service operation is not there when the corresponding service personnel 8 is not there, any other service personnel 8 can be further transferred to themselves through the manual matching device 206 to realize real-time for service.

When having this customer 9 of foreign nationality to use this front desk equipment 1, and when making a voice inquiry with its customary language, can refer to as shown in Fig. Judging the language of the received voice information, the synthetic human voice generated accordingly will be able to respond to the customer in the corresponding language 9 . Further, the language matching device 203 can automatically match to the corresponding voice-tuned microphone 21 after judging the language of the voice information, so that the service personnel 8 who have the professional ability of the language can perform services, and use the second 5, the attendant 8 in the middle position makes the loudspeaker 12 play the modulated human voice through the voice-modulated microphone 21. Of course, the language identifier 105 can also be defined with the language identification word, and the language identification word can be included in the intelligence information to be displayed to the customer through the dynamic screen D, so that the customer 9 can first Speak the language identification word in its customary language, so that the front host 10 can quickly identify the language type and perform subsequent actions.

To sum up, the service system of the present invention utilizes the actions of central control entities to synchronize the "virtual audio and video of distributed sites". From the perspective of enterprises, the allocation of customer service resources at distributed sites requires less than The voice-tuning microphones for the number of front-end equipment, as well as the dynamic images of virtual characters on each front-end equipment, realize the effect of allowing one person to serve customers at multiple sites, thereby eliminating the need to allocate real waiters to each site one by one in the past high cost.

From the perspective of customers, the present invention uses the synthesized human voice and the modulated human voice with the same timbre and the same audio frequency range to achieve the effect of seamlessly switching between real people and artificial intelligence, so that customers will not suffer from switching between service objects during use. Unobtrusive, but with a consistent use experience. For example, for elderly users or hearing-impaired persons, there is no need to worry about being unable to use it smoothly. The present invention allows the service personnel's actions to be displayed in real-time and synchronously with the virtual character information for the customer to watch, or further through manipulating the Dynamic pictures are used to enhance information display, so if it is not conducive to the customer who listens to the synthesized human voice or the modulated human voice, he can understand and obtain the required information through visual representation. And further, when the service system has the language recognizer, it can solve the difficulties of foreign customers in the past when asking or seeking help, as long as it can be used easily in the customary language.

As far as service personnel are concerned, the real-time audio and video of customers captured by the front-end equipment and the audio-visual playback device in the background can better and comprehensively respond to the status of each customer and provide the required service. Furthermore, through the structure of the service system, the service personnel can focus more on replying and providing the needs of the customers, so as to provide better service quality. For example, the speaker can be automatically switched to receive the modulated human voice output through the authority switcher, and the customer's problem can be dealt with in real time, so as to prevent the customer from being unhappy because the content of the synthesized human voice does not meet the requirements. Or the automatic matching device can be used to evenly distribute the work times of each of the voice-tuned microphones, that is, the service personnel, so as to prevent the unfair situation of uneven workload caused by human factors. On this basis, the service system can also have manual matching that takes precedence over automatic matching The configuration function can be used to avoid the situation that the service personnel cannot deal with it in real time after the automatic allocation, which will cause customer displeasure. At the same time, the automatic matching device can also allocate the number of times of work according to the preset value, and after the number of times of work is reached, it will be opened for each service personnel to manually undertake the service operation, and a reward system can also be formed to encourage each service person. personnel.

In general, the present invention is based on the idea of providing all customers with a warm and emotional humanized interactive experience even when communicating with virtual characters in dynamic images, and based on the brand-new service conceived and planned under the high-speed network environment system structure. In other words, the present invention proposes the concept of central control, allowing distributed sites to replace physical personnel with virtual audio and video that is synchronized with real people in the central control, forming a brand-new service system that crosses the boundary between virtual and real worlds architecture.

The above is only a description of the preferred embodiment in the patent scope of the present invention, but not to limit the scope of rights of the present invention based on the content of this embodiment; therefore, the textual changes made without departing from the equivalent scope of the present invention or modifications, should still be covered within the scope of the patent application of the present invention.

10: Foreground host

101: Audio-visual recording device

11: Glass Information Desk

12:Speaker

2: background equipment

20: Central control server

204: Site marker

21: Tuned microphone

22: Audio-visual playback device

23:Motion Capture

8: Service staff

A: Internet

Claims (8)

A service system for synchronizing "virtual audio and video in distributed sites" by using central control entity personnel actions, including: a plurality of front-end equipment, including: a front-end host with an audio-visual recording device for capturing at least one customer's A plurality of sound information, including: an image generator for generating a virtual character information; a semantic analyzer for receiving the sound information, and calculating and judging the sound information through an artificial intelligence engine on the machine or in the cloud to form a Automatic response information; and a human voice simulator, connected by telecommunication to the semantic analyzer, for receiving the automatic response information and generating a synthetic human voice; a glass information platform, part or all of which is a transparent display, connected by telecommunication to the image generator, And receive the information of the virtual character and display a dynamic picture accordingly; and a loudspeaker, connected to the human voice simulator by telecommunications, for receiving the output of the synthesized human voice; At least one service person operates on the real-time audio and video of the customer captured by the audio and video recording device, including: at least one central control server, which is telecommunications connected to the front host, and has an authority switcher, and the authority switcher is connected to the Loudspeaker telecommunications connection; at least one voice-tuning microphone, telecommunications connected to the central control server, for receiving a manual response information input by the service personnel, and generating a modulated human voice according to the manual response information modulation for the speaker to output; At least one audio-visual playback device, connected to the central control server by telecommunications, for displaying the real-time video and audio of the customer captured by the audio-visual recording device, as well as the dynamic picture; and at least one motion capture device, connected to the central control server by telecommunications The device and the audio-visual playback device are used to capture a behavior of the service personnel, and use the behavior to drive the virtual character information into a consistent action; wherein, the number of the front-end devices is N, and the number of the voice-tuned microphones is M, and N>M; in addition, the authority switcher automatically or manually switches the authority of the speaker to receive the synthesized vocal or the modulated voice, and the synthesized voice and the modulated voice have the same timbre and the same audio range; The central control server further includes: a point marker, an automatic matching device and a manual matching device. When the number of the voice-tuning microphones is more than two, the automatic matching device is connected to the voice-tuning microphones by telecommunications. Evenly distribute the working times of these voice-tuning microphones, and the location markers display the location of the front-end equipment, and the connection status between the front-end equipment and the voice-tuning microphones; the manual matching device, telecommunications connection The voice-tuned microphones give priority to the automatic matching device to force work matching, wherein the automatic matching device performs automatic allocation according to a preset value, and when the voice-tuned microphones reach the preset value, the automatic allocation is excluded . The service system as described in Claim 1, wherein the audio-visual playback device has a touch screen, and the touch screen is used by the service personnel to input at least one gesture control, and accordingly the dynamic picture is executed synchronously. The service system as described in Claim 1 or 2, wherein the image generator further generates information information, and the glass information desk receives the information of the virtual character and the information information, and simultaneously displays the information containing the information of the virtual character and the information information The dynamic picture. The service system as described in Claim 1, wherein when the content of the automatic response information repeats beyond a threshold value, the authority switcher automatically switches the speaker to receive the modulated human voice output. The service system as described in Claim 1, wherein the front-end host computer further includes: a language identifier connected to the semantic analyzer by telecommunication for judging the language of the voice information. The service system as described in claim 5, wherein the language recognizer stores a defined language recognition word for the customer to use the language recognition word as a pronunciation expression, and use the sound information as a basis for language recognition . The service system as described in claim 5 or 6, wherein the central control server further includes: a language matcher, which is connected to the language recognizer by telecommunications, and automatically matches to the corresponding language after judging the language of the voice information It should be a voice-tuned microphone. The service system as described in claim 1, wherein the central control server further includes: at least one text compiler connected to the semantic analyzer by telecommunication for sequentially compiling the voice information and response information for the customer into text The information is displayed through the audio-visual playback device.

Images