JP6266330B2 - Remote operation system and user terminal and viewing device thereof - Google Patents

Description

  The present invention relates to a system for remotely operating a viewing device such as a TV, a set top box (STB), a car navigation system, or a digital photo frame on a viewer's mobile terminal (user terminal), and the user terminal and viewing device thereof. In particular, the present invention relates to a remote operation system that can be remotely operated in a unified manner without making the user aware of differences in operation target terminals by using a character interactive UI, and the user terminal and viewing device.

  Infrared remote controls are widely used as devices for remotely operating viewing devices such as televisions. However, in the infrared remote control, when the light emitting unit is not directed to the light receiving unit of the viewing device, when the light receiving unit is exposed to strong illumination light such as a fluorescent lamp, there is an obstacle between the remote control and the viewing device In some cases, the operation may become unresponsive. In addition, the operation becomes complicated as the functions of the remote control become higher, and the button positions and operation methods of the remote control are not standardized for each viewing device, which may cause confusion for users who operate a plurality of viewing devices.

  On the other hand, in recent years, Wi-Fi and Bluetooth (registered trademark) have been installed in viewing devices, and it has become possible to link with user terminals such as smartphones and tablet terminals.

  Patent Document 1 discloses a technique for recognizing a voice uttered by a user and converting it into a control code for a viewing device.

  Patent Document 2 discloses a playback technique using a mobile remote controller that uses a Bluetooth (registered trademark) communication system to link the playback time of content between a mobile phone and a viewing device.

  Patent Document 3 discloses a technique in which a screen area of a viewing device is stored in association with a mobile phone, and when the mobile phone is wirelessly connected to the viewing device, the assigned screen area can be operated from the mobile remote controller.

  Patent Document 4 discloses a universal remote control technology that can be used without performing difficult operations on different devices such as a television, a video play, a MAC computer, and a tablet.

JP 2006-350221 A JP 2009-43309 JP 2009-27485 JP United States Patent Application No.20120019371

  In Japanese Patent Laid-Open No. 2004-26883, the audiovisual device can be remotely controlled by speaking the power on / off, playback, fast-forwarding, etc. to the remote control, but what operation can be performed on which screen and what voice command should be spoken It is necessary for the user to remember whether or not it is good.

  In Patent Documents 2 and 3, a viewing device can be remotely controlled from an application using a wireless communication method such as Wi-Fi or Bluetooth (registered trademark), but a UI that can be easily and uniformly operated regardless of whether the viewing device is the same. Is difficult to realize.

  Patent Document 4 is a technology of Apple Inc. that has the strength to develop the whole from hardware to software and operating system, but as with Patent Documents 2 and 3, regardless of the difference in viewing equipment. Realizing a uniform and easy-to-operate UI is not easy. Actually, there are many differences between the UI of the iPhone (registered trademark) remote control for operating Apple TV and the UI of the iPad (registered trademark) remote control, and some users with low IT literacy felt confused.

  Furthermore, each of the above prior arts is still in the category of remote control operation, how to operate various devices in a unified and easy manner, functions that take into account user lifestyles and preferences across different viewing devices. Issues such as how content recommendation can be realized remain.

  A first object of the present invention is to move a character that virtually interacts with a user between displays of each device in response to switching of the operation target device, and can request remote operation in an interactive manner with the character. By doing so, it is to provide a remote operation system that allows a user to remotely operate each device by a unified method without being aware of the difference between viewing devices.

  The second object of the present invention is to reflect the user profile registered in the user terminal in the response content when each viewing device responds to a remote operation from the user, so that each viewing device has the user's preference and It is an object to provide a remote control system capable of performing a response suitable for a lifestyle, a user terminal thereof, and a viewing device.

  In order to achieve the above object, the present invention has the following configuration in a remote operation system in which a character is moved from a user terminal to an operation target viewing device on a display and remotely operated by a character interactive UI. There is a feature.

  (1) The user terminal comprises semantic understanding means for understanding the user's utterance content, and means for providing the utterance content to the viewing terminal,

  A user terminal and a viewing device, a wireless communication means for wirelessly connecting terminals; an interactive response means for determining a response content based on a user's utterance content; a voice response means for outputting a voice message based on the response content; First animation control means for controlling the animation of the character on the display according to the content of the response, and second animation control means for jumping out the character from the user terminal and jumping in to the viewing device to be operated, The viewing device further includes control means for controlling the viewing service based on the response content.

  (2) The user terminal includes means for storing the user profile, and the dialog response means determines the response content reflecting the user profile.

  (3) The user terminal provides the user profile to the viewing terminal that jumps in the character.

  (4) The interactive response means of the viewing terminal determines the response content reflecting the user profile provided from the user terminal.

  According to the present invention, the following effects are achieved.

  (1) One character that virtually interacts with the user moves between the displays of each device in response to the switching of the viewing device to be operated and appears on the display of the operation target device. Regardless of the device, remote operation can be requested in an interactive manner with the character displayed on the display. Therefore, each device can be remotely operated by a unified method without making the user aware of the difference between the viewing devices.

  (2) When the character moves from the user terminal to the viewing device, the profile including the user's preferences is also notified to the viewing device, so each viewing device reflects the user profile in the requested remote operation and the response content Can be determined.

  For example, when a remote operation for switching a TV channel being viewed is detected, if a profile indicating that the user likes sports broadcasting is acquired, channel switching to a sports program can be proposed or prioritized. .

It is the figure which expressed the outline | summary of this invention typically. It is a functional block diagram of a remote control system to which the present invention is applied. It is the sequence flow which showed the operation | movement of the principal part of FIG. 2 in the remote control of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, first, an outline of a method of remotely operating a viewing device in conjunction with the user terminal 1 by using the character interactive UI of the present invention will be described with reference to the schematic diagram of FIG.

  When a remote operation application (hereinafter referred to as a remote application) is activated on the user terminal 1 (here, a smartphone is assumed), a character imitating a woman appears on the terminal display as shown in FIG. It is displayed superimposed. The remote application searches for TV program programs based on pre-registered profile information such as user interests and preferences, and if a program program that matches the user interests and preferences is found, for example, “XX-kun (user A voice message such as “It ’s time for your favorite professional baseball broadcast” is synthesized and uttered in a pseudo manner from the character.

  Here, when the user utters “Turn on!”, “TV ON”, “I want to watch this program”, the voice is detected by the microphone of the user terminal 1 and transferred to the voice recognition unit 103, and the recognition result. Is transferred to the semantic understanding unit 104. The meaning comprehension unit 104 recognizes the user's utterance content as a remote operation request to turn on the switch of the viewing device 2 (assuming TV here), so that the user terminal 1 is for remote control to turn on the TV 2. A signal is generated and transmitted to the TV 2.

  Since the remote operation application knows the channel of the professional baseball broadcast recommended for viewing by the user, here, the “switch-on operation” and “channel designation operation” control signals suitable for the TV 2 are generated and sent to the TV 2. Sent.

  In the TV 2, as shown in FIG. 1 (b), the switch in response to each control signal is turned on, and the channel is switched to the designated channel to display a menu screen including professional baseball broadcasts.

  Further, the character jumps out from the display of the user terminal 1 and jumps into the display of the TV 2, and the control target is switched from the user terminal 1 to the TV 2 in synchronization with the movement of the character. At this time, the user profile is also provided from the user terminal 1 to the TV 2.

  Here, for example, when the user utters “Don't lose. What other programs are?”, This is detected by the microphone of the user terminal 1, voice recognition is performed, and the result of the voice recognition is transferred to the TV 2. Since the TV 2 determines that the request for recommending another program program is based on the result of the voice recognition, another program program that matches the user's interests and preferences is being broadcast based on the provided user profile. Whether or not there is is determined by referring to the program guide.

  When it is understood that a soccer game is being relayed on another channel, the start time “7:30” and the content “Japan National Team” are acquired from the program guide as shown in FIG. The voice is synthesized and, for example, a voice message saying “It is a soccer match against Japan from 7:30” is uttered from the character.

  When the user responds to this voice message, for example, “Take it”, the voice is detected by the microphone of the user terminal 1, voice recognition is executed, and the result of the voice recognition is transferred to the TV 2. Since the TV 2 recognizes that the channel switching to the soccer relay has been approved based on the result of the voice recognition, the channel is switched to the soccer relay channel. As a result, as shown in FIG. 4D, a soccer broadcast is displayed on the display of the TV 2 instead of the baseball broadcast.

  After that, when the soccer game ends and the TV program end time approaches, the character appears again as shown in FIG. Even when a TV program is being reproduced, the character appears when the user generates and calls the character name, name, nickname, or the like. Here, when the user utters, for example, “Tweet, I did it! Congratulations!”, This is detected by the microphone of the user terminal 1, voice recognition is executed, and the result of the voice recognition is transferred to the TV 2.

  Since the TV 2 recognizes the tweet request based on the result of the voice recognition, the character jumps out from the display of the TV 2 and simultaneously jumps onto the display of the user terminal 1 to return the operation target from the TV 2 to the user terminal 1. In.

  In the user terminal 1, a tweet application is activated, and the message is voice-recognized and converted into text, which is input to the message input field of the tweet application. When the text input is completed, as shown in FIG. 5 (f), the character is displayed together with the input content. As a message for obtaining approval of the input content, for example, a voice message “Is this OK?” Is sent from the character. Spoken.

  In response to this question, for example, when the user responds with a voice saying “OK”, this is detected and recognized by the microphone of the user terminal 1, and if it is determined to be approved, the sweet is transmitted to a predetermined address.

  When the user wants to switch off the TV, when the user speaks “Close TV”, “TV OFF”, “I ’m going to go to bed now because I ’m tired”, the voice is detected by the microphone of the user terminal 1. The recognition result is transferred to the voice recognition unit 103, and the recognition result is transferred to the meaning understanding unit 104. The meaning comprehension unit 104 recognizes the user's utterance content as a remote operation request for turning off the switch of the viewing device 2, so that the user terminal 1 generates a remote control signal for turning off the TV 2 and transmits it to the TV 2. Is done.

  As described above, in the present invention, in order to operate and link a plurality of types of information devices including the user terminal in a unified manner, one character that virtually interacts with the user with movement is responded to the switching of the operation target device. First, the user can easily recognize the device selected as the remote operation target by adopting a character interactive UI that conveys information by moving it between the displays of each device together with various information. Secondly, unified operability is realized in which the user is not aware of the difference between the operation target devices.

  FIG. 2 is a block diagram showing the configuration of the main part of the audiovisual device control system according to one embodiment of the present invention, and here, the configuration unnecessary for the description of the present invention is omitted. In this embodiment, paying attention to the STB as a viewing device, the TV 2 is connected to the STB 3, the display function is borne by the TV 3, while the viewing device functions other than the display function are borne by the STB 3, and the STB 3 is linked to the user terminal 1 for dialogue. A case where remote control is performed by the method will be described as an example.

  In the user terminal 1, the user profile storage unit 101 stores a terminal ID (such as a MAC address or a mobile phone number) unique to the user terminal as a user profile, and further includes name, age, gender, hobby as user attribute information. , Preferences, favorite programs, and actor names of nieces are stored.

  The wireless communication unit 102 establishes wireless connection with the wireless communication unit 301 of the STB 3 by Wi-Fi, Bluetooth (registered trademark), etc., understands the user's speech, the terminal ID unique to the user terminal, the user Profile information including the name and age of the user, the user's preferences, etc., the execution data of the character interactive UI, etc. are wirelessly transmitted to the STB 3.

  The voice recognition unit 103 and the meaning understanding unit 104 recognize the terminal user's voice detected by a microphone (not shown) and understand the user's request from the utterance content. The interactive response PF 105 actively asks the terminal user a question or generates an answer sentence to the request from the terminal user. Inside the dialogue response PF 105, a chat dialogue pattern of daily life of the terminal user and a table of state transition are registered in advance.

  The character display unit 106 and the voice synthesizing unit 107 realize human-like and natural conversation by displaying an anthropomorphic character animation and voice synthesis. The speech synthesizer 107 further has a function of converting text such as an answer sentence generated by the dialogue response PF 105 into speech.

  The character display unit 106 includes a first animation control unit 106a that controls the animation of the character on the display according to the response content, and a second animation control unit 106b that jumps out and jumps in the character.

  In STB3, after the character moves from the user terminal 1 to the STB3, the dialogue response PF302 actively asks the terminal user a question or generates a response to the request from the terminal user.

  Similarly to the user terminal side, the dialog response PF 302 includes a chat dialog pattern and state transition table of the terminal user's daily life, and is associated with a viewing request analyzed from the dialog with the user. STB3 device operation control codes (channel switching, volume adjustment, application activation, etc.) are registered.

  The character display unit 303 and the voice synthesizing unit 304 realize human-like natural conversation by displaying an anthropomorphic character animation and voice synthesis. The voice synthesizer 304 further has a function of converting text such as an answer sentence generated by the dialogue response PF into voice.

  The program search unit 305 identifies the user terminal 1 and refers to the rating information (viewing restriction information) of each content corresponding to the user attributes (terminal ID, name, age, preference information) of the user terminal 1. Then, it is determined whether or not the user satisfies the rating information of the requested content, and if the content is satisfied, reproduction of the content is permitted to, for example, a VOD (Video On Demand) service unit 306. The rating information includes R20 that prohibits viewing under the age of 20, R18 that prohibits viewing under the age of 18, and R15 that prohibits viewing under the age of 15. The application unit 307 manages applications provided by third parties such as YouTube (registered trademark), karaoke, and dictionaries. The control unit 308 controls the viewing service based on remote operation.

  Next, the animation effect of the character on the character display units 106 and 303 will be described.

  In this embodiment, each device has a similar character display, speech synthesis, and interactive response execution framework. In order to realize efficient and continuous character movement and information presentation, only the 3D model file, motion file, and text file for dialogue necessary for character execution need be transferred. Further, since these transfer data are in a text format, transmission / reception delay is small.

  In this example, the Miku Miku Dance (MMD: 3DCG movie production tool) format is adopted for the 3D model file and motion file, and when the drawing is linked to the 3D model file, various 3DCG animation with various combinations can be realized. This 3D model file is created by 3D polygon modeler software, and can create and edit solid objects in units of polygons.

  The motion file is a text file obtained by actually taking sampling information of human movements using dedicated equipment and software for motion capture. Actually, it is often used to reproduce human-like movements of characters in computer animations such as movies and games. The data of the motion file describes the skeleton of the model similar to the 3D model file and the skeleton / joint difference information for each frame. By drawing 30 frames per second at the time of execution, natural motion can be expressed continuously.

  Further, in this embodiment, a regular speech synthesis technique is used for speech synthesis for generating text information on a character. Mobile terminals have limited processing capacity and memory capacity, and the database capacity of the voice model cannot be secured sufficiently. Therefore, when using the voice reading function, it is necessary to have the server process through a network such as a cellular phone line.

  Therefore, in this embodiment, the HMM speech synthesis method that can make the voice quality data smaller is adopted, and the parameter that determines the behavior of the HMM by giving the paired data of text and speech data to the statistical model called HMM Is generated, and parameters necessary for speech synthesis are generated by giving text data to the learned HMM.

  With this lightweight technology, this example can generate natural audio content from text even on STBs, smartphones / tablets, in-vehicle devices with insufficient processing capacity and memory capacity, and enables real-time information reading and narration creation. Become.

  Next, a description will be given of a character movement expression between a plurality of devices (user terminal / STB) by the second animation control unit 106b of the character display unit 106. FIG.

  In this embodiment, in order to realize continuous movement between displays such that a character jumps out from one display A and jumps into the other display B at the same time, two displays A and B are virtually combined into one. Treated as a drawing area.

  For example, when a part of the character (for example, the head) jumps out of display A, only the head of the character is displayed on display B, and when the body jumps out of display A, the body jumps to display B In.

  Such character synchronization can be realized by synchronizing the frames of the motion file of the character jump-out effect of the user terminal 1 and the character jump-in effect of the STB 3.

  In the user terminal 1, while drawing motion frames for character jump-out effects one by one on the screen, the Sync command is transmitted to the STB 3 together with the frame ID being drawn. When STB3 receives the Sync command, it analyzes the frame ID and draws it on the screen of the television using the motion frame ID of the character jump-in effect corresponding to it.

  Next, functional operations of the user terminal and the STB will be described before and after the character movement. The user interacts with the character on the display of the user terminal 1 on a daily basis, and when the user terminal 1 detects a television viewing request from the user, the user terminal 1 wirelessly connects to the STB 3 to jump in the character to the screen of the TV 2. At this time, the user terminal 1 becomes the microphone of the TV 2 (for voice input), and the user's utterance is converted into text by voice recognition and meaning understanding, and transferred to the dialogue response PF 302 of the STB 3.

  Thereafter, the dialog response PF 302 of the STB 3 estimates the user's intention to operate, and the character performs visual feedback and voice response and executes the equipment operation of the STB 3. By building an engine that stores visual data / speech synthesis models of the same or equivalent character on user terminal 1 and STB 3, only text information is passed between user terminal 1 and STB 3. Realistic character interactive UI can be realized.

  Next, packet structures of various messages transmitted / received between the user end 1 and the STB 3 will be described. In this embodiment, it is assumed that devices are wirelessly connected by using TCP / IP Socket communication, and a packet is composed of fields of HEADER, CMD, PARAM, END, and SUM.

  A start mark is registered in HEADER. Execution instructions (commands) are registered in the CMD. PARAM includes a plurality of Value fields. An end mark is registered in END. A checksum for checking the integrity of the message is registered in the SUM field.

  For example, in a connection request message transmitted from the user terminal 1 to the STB 3, a command corresponding to “user verification” is registered in the CMD field, and user attributes (here, name, age, preference information, etc.) are registered in the PARAM field. And the terminal ID are registered.

  In addition, if the message understands the meaning of the user's utterance, a command corresponding to the “control code” (here, opening / closing of the TV, program search, channel switching, etc.) is registered in the CMD field, and in the PARAM field, Keywords of user utterances, parts of speech (nouns, verbs, place names, actor names, etc.) and terminal IDs (here, terminal manufacturing IDs) of each keyword are registered.

  For example, when executing a command for searching for a program, the program guide is searched using each keyword analyzed from PARAM. In the search of the program guide, a narrow search such as program contents, actors, and categories can be performed.

  Next, the function of the dialogue response PF 105 (302) will be described. The dialogue response PF 105 (302) is a platform for interacting with a user based on a dialogue scenario.

  An interaction scenario is composed of one or more state nodes, and each state node executes a respective interaction pattern. For example, when a user listens to a program being broadcast to a character at the first state node, the character makes a recommendation according to the user's preference and moves to state node 2. When the user speaks in the state node 2 to see the recommended program, the power of the STB 3 is turned on, and the character jumps from the user terminal 1 onto the screen of the TV 2 to move to the state node 3.

  In the state node 3, commands such as switching of other channels, searching for a TV program guide, starting a VOD content application, and other applications 307 such as YouTube (registered trademark) and karaoke are accepted while the program is being played. Here, for example, when the VOD content application is activated, the process moves to the state node 4 and waits for the utterance of the search keyword from the user.

  The state node of the dialogue scenario and the transition between each state node are prepared by creating a state node transition diagram based on the actual viewing use case statistics. In order to respond accurately by user input, general and basic state nodes and transition rules are first created from a collection of viewing-related cases of a large number of users. Then, by gradually repeating the addition / modification of the pattern of the state node and transition rule, it is possible to improve the dialogue accuracy related to various viewing operations of the user.

  Next, viewing operations and content recommendation based on user attributes will be described. In the STB 3, when a connection request message transmitted from the user terminal 1 is detected, the terminal ID and the user profile are extracted from the message and stored in the memory. In the subsequent dialogue, it is determined whether or not the viewing operation requested by the user is subject to regulation. If the request is unrelated to the rating, such as volume control or brightness control, control according to the request is executed. .

  On the other hand, if the request is a content viewing request for which rating is set, the rating of the requested content is read from the program guide, and the user profile (here) is associated with the extracted terminal ID Then, age) and rating information are compared. If the user age is not the restriction target, viewing is permitted, while if the user age is the restriction target, the viewing is rejected.

  The user profile storage unit 101 of the user terminal 1 stores the preference information of the user. When the connection between the user terminal 1 and the STB 3 is established, the preference information is transferred to the STB 3 together with the character information. And used for program search and content recommendation.

User preference information includes favoritetvprogram (favorite program name), favoritetvgenre (favorite category), favoritetetalent (favorite actor name), favoriteplace (favorite place), favaritesports (favorite sports), for example: Such information is linked.
favoritetvprogram / You can laugh / Refresh
favoritetvgenre / News / Documentary / Animation
favoritetetalent / Seiji Miyane / AKB / Eiichiro Funakoshi
favoriteplace / Tokyo / Korea
favaritesports / baseball / golf

  Next, an intention estimation method for various expressions of users will be described. In this embodiment, the user's operation intention is captured more accurately by estimating the intention of the dialog rather than simply matching the keyword spoken by the user.

  Specifically, the meaning understanding unit 104 receives a character string representing an utterance and outputs a symbol representing an intention called an intention slot. Each intention slot contains multiple templates that convert utterances that are assumed to belong to the intention into feature vectors, and calculates the similarity between the feature vectors converted from the input utterance character string and each template. The intention slot to which the template having the highest similarity belongs is output.

  As a specific algorithm, we extract a set of content words abstracted from synonyms and numerical expressions from utterance strings, and the dimension corresponding to the content words defined in the template dictionary is non-zero as a bag-of-words expression. A feature vector of size “1” is created. Using this feature vector, the similarity with each template in the template dictionary is calculated, and the intention slot symbol to which the template with the highest similarity belongs is output as the intention estimation result.

  Next, procedures for automatic discovery and automatic connection of the STB 3 by the user terminal 1 will be described. Generally, since the IP address of STB3 is acquired by the DHCP of the CATV provider or the local router, it is difficult to uniquely identify it. Therefore, the present invention introduces a mechanism for notifying the user terminal 1 of the IP address of the STB 3.

  In this embodiment, the user terminal 1 connected to the local network performs a broadcast search via UDP, and the STB 3 returns the IP address and communication port assigned to itself. The user terminal 1 automatically requests connection to the STB 3 using the connection information such as the returned IP address and communication port. As a result, the terminal user analyzes the IP address of the STB 3 and is further freed from the operation of manually connecting based on the analysis result.

  FIG. 3 is a sequence flow showing the operation of the main part of FIG. 2 in the remote operation of FIG. 1. The meaning understanding unit 104 of the user terminal 1 recognizes the audio signal for turning on the switch of the TV 2 / STB 3. At times t1 and t2, a power ON signal is transmitted from the dialogue response PF 105 to the wireless communication unit 301 of the STB 3 via the wireless communication unit 102. At time t3, an ACK signal (power ON completion) is returned from the wireless communication unit 301 of the STB 3 to the user terminal 1.

  At times t4 and t5, the character data (3D model file and motion file necessary for displaying the character) necessary for displaying the character on the display of the TV 2 and performing various effects are displayed in the dialog of the user terminal 1. The response is transmitted from the response PF 105 to the wireless communication unit 301 of the STB 3 via the wireless communication unit 102.

  At times t6 and t7, an ACK (information transmission completion) for the character data is returned from the wireless communication unit 301 of the STB 3 to the dialogue response PF 105 via the wireless communication unit 102 of the user terminal 1. In parallel with this, at time t8, the character data is transferred from the wireless communication unit 301 of the STB 3 to the dialogue response PF 302.

  Thereafter, when a jump-out drawing request is transmitted from the dialogue response PF 105 of the user terminal 2 to the character display unit 106 at time t9, a jump-out display of the character is produced on the terminal display.

  At time t10, the dialog response PF 105 notifies the wireless communication unit 102 of the completion of jump-out. At times t11 and t12, the jump-out completion is transmitted from the wireless communication unit 102 to the dialogue response PF 302 via the wireless communication unit 301 of the STB 3. At time t13, the jump-in drawing request is transferred from the interactive response PF302 of STB3 to the character display unit 303, and a character jump-in display is produced on the TV2.

  In the above embodiment, the case where the viewing device is an STB has been described as an example. However, the present invention is not limited to this, and the display device such as a car navigation system or a digital photo frame is wirelessly provided. As long as the device can be operated remotely, it can be applied to any viewing device.

DESCRIPTION OF SYMBOLS 1 ... User terminal, 2 ... TV, 3 ... STB, 102, 301 ... Wireless communication part, 103 ... Voice recognition part, 104 ... Semantic understanding part, 105, 302 ... Dialog response PF, 106, 303 ... Character display part, 107 , 304 ... Voice synthesis unit, 305 ... Program search unit, 306 ... VOD service unit, 307 ... Application unit, 308 ... Control unit

Claims (10)

The display of the character interactive UI character that virtually interacts with the user is moved between the displays of the user terminal and the viewing device, and the side of the user terminal and the viewing device on which the character is displayed is displayed. A remote control system that operates interactively with the characters inside,
The user terminal is
Meaning understanding means to understand user's utterance content,
Providing the utterance content to a viewing terminal,
The user terminal and the viewing device are
Wireless communication means for establishing a wireless connection with each other;
Dialogue response means for determining response content based on the user's utterance content;
Voice response means for outputting a voice message based on the response content;
First animation control means for controlling the animation of the character on the display in accordance with the response content;
Second animation control means for jumping out the character from the user terminal and jumping in the viewing device to be operated;
The viewing device further comprises:
A remote operation system comprising control means for controlling a viewing service based on the response content. The user terminal comprises means for storing a user profile;
The remote operation system according to claim 1 , wherein the interactive response unit determines a response content by reflecting the user profile. The remote operation system according to claim 2 , wherein the user terminal provides the user profile to a viewing terminal that jumps in a character. 4. The remote operation system according to claim 3 , wherein the interaction response means of the viewing terminal determines the response content reflecting the provided user profile. The second animation control means, remote control system according to any one of claims 1 to 4, characterized in that to jump-in to the user terminal by jumping out from the viewing device to which the operation is completed the character. In a user terminal of a remote control system in which a character is moved from a user terminal to a target viewing device on a display and remotely controlled by a character interactive UI,
Meaning understanding means to understand user's utterance content,
Means for providing the utterance content to a viewing terminal;
Wireless communication means for establishing a wireless connection with the viewing device;
Dialogue response means for determining response content based on the user's utterance content;
Voice response means for outputting a voice message based on the response content;
First animation control means for controlling the animation of the character on the display in accordance with the response content;
A user terminal of a remote control system, comprising: second animation control means for jumping out the character in synchronization with jump-in to the viewing terminal. Means for storing a user profile;
The user terminal of the remote operation system according to claim 6 , wherein the dialog response unit determines a response content by reflecting the user profile. 8. The user terminal of a remote control system according to claim 7 , wherein the user profile is provided to a viewing terminal to which the character has jumped in. In a viewing device of a remote control system in which a character is moved on a display from a user terminal to a target viewing device and remotely controlled by a character interactive UI,
Wireless communication means for establishing a wireless connection with the user terminal;
A dialog response means for determining a response content based on a user's utterance content provided from a user terminal;
Voice response means for outputting a voice message based on the response content;
First animation control means for controlling the animation of the character on the display in accordance with the response content;
Second animation control means for causing the character to jump in in synchronization with a jump-out from a user terminal;
A viewing device of a remote control system, comprising: a control unit that controls a viewing service based on the response content. Means for obtaining a user profile from a user terminal;
The viewing apparatus of the remote operation system according to claim 9 , wherein the interaction response means determines the response content by reflecting the user profile.