JP2023147653A - System, interface apparatus, program, and the like - Google Patents

Abstract

To provide a technique different from the conventional one, which is a system or the like that realizes an interface of a processing apparatus with a different apparatus.SOLUTION: A system includes a processing apparatus and an interface apparatus communicatively connected to each other. The processing apparatus includes a control unit that performs control to execute processing according to an input from a user to the interface apparatus. The interface apparatus includes a reception unit that receives an input, and an output unit that outputs information according to the input or the processing to the user with association with a predetermined character.SELECTED DRAWING: Figure 1

Description

The present invention relates to a system, an interface device, a program, etc.

External devices that can control processing devices such as computers are known. For example, inventions in which a user can control a processing device by speaking to an external device are described in Patent Document 1, Patent Document 2, and Patent Document 3.

JP 2021-73553 Publication Japanese Patent Application Publication No. 2002-41276 Unexamined Japanese Patent Publication No. 2019-66702

One of the purposes of the present disclosure is to provide a system or the like in which an interface of a processing device is realized by a different device, and a technology different from the conventional technology.

The purpose of the present invention is not limited thereto, and the present invention intends to acquire rights through divisional applications, amendments, etc. for structures that aim to obtain effects from the parts of the structures disclosed in this specification, drawings, etc. For example, the present specification discloses a problem in which passages such as ``can be done'' or ``possible'' are read as ``the problem is.'' Each issue is described as an independent entity, and we intend to acquire rights to the structure for solving each issue individually through divisional applications, amendments, etc. Even if the problem is implicitly understood from the description of the specification, the present applicant has the intention to claim a part of the structure described in the specification in an amendment or divisional application. The company also discloses a structure that solves a problem that combines these independent problems, and has the intention to acquire the rights to it.

(1) In view of the above-mentioned problems, the system of the present embodiment includes a processing device and an interface device that are communicably connected to each other, and the processing device performs processing according to user input to the interface device. The interface device includes a reception unit that receives the input, and an output unit that outputs information corresponding to the input or the processing to the user by associating it with a predetermined character. and may have.

In this way, by inputting input to the interface device external to the processing device, the user can cause the processing device, which is a device different from the interface device, to execute processing according to the user's input, and also Alternatively, information corresponding to the executed process can be recognized through the interface device by associating it with the character. For example, a user can use an interface device to remotely control a processing device and understand the results of the remote control through the presentation of a character on the interface device. You can get more fun using it without having to worry about it.

Inputs that can be accepted by the interface device are particularly preferably inputs that can be intuitively performed by the user, such as inputs of voice uttered by the user, inputs of gestural movements shown by the user, etc. In this way, the user's input burden on the interface device can be reduced.

The processing executed by the processing device is preferably a process executable by the processing device, and particularly preferably a process executed by a function of software installed in the processing device. The software may be an OS or software running on the OS. In this way, the user can execute a process that can be executed by the functions of the OS installed in the processing device or the software running on the OS by inputting it to the external interface device. Since the user does not have to operate the processing device to instruct the OS, the software running on the OS, or the processing to be executed by the OS, the input burden on the user can be reduced.

The output unit preferably outputs user-recognizable information in association with the character, and particularly preferably outputs an image or voice of the character. In this way, the user can easily recognize the information that is output in association with the character and that corresponds to the user's input or processing. The information corresponding to the process input or executed may be information indicating that the input has been received or an instruction to execute the process, but it is particularly preferable to use information that allows the user to recognize the result of the process. In this way, the user can recognize the results of the processing performed by the processing device through the characters displayed on the interface device rather than through the processing device.

(2) The output unit may output information according to the input or the processing by displaying an image of the character that changes according to the input or the processing.

In this way, the user can visually recognize the image of the character that changes according to the user's input or the process instructed to be executed by the user, so that the user can obtain visual interest through the character. Can be done. Such a visual effect makes it easier for the user to feel familiar with the character, which in turn provides a strong motivation to use the interface device to instruct the processing device to execute a process. In this case, it is particularly preferable that the input that the reception unit accepts is the voice input from the user. In this way, the user can feel as if he or she is having a face-to-face conversation with the character, and the user can even more strongly feel as if he or she is remotely controlling the processing device while communicating with the character. It can be made easier to obtain.

(3) The image of the character that changes depending on the information according to the input or the processing may be an image in which the facial expression of the character or the action of the character changes.

In this way, the user can visually recognize the image of the character whose facial expressions or actions change according to the user's own input or the processing instructed to be executed by the user, so that the user can feel as if he or she is communicating with the character. You can get stronger.

(4) The output unit may output information according to the input or the processing using the voice of the character that changes according to the input or the processing.

In this way, the user can hear the character's voice that changes depending on the user's input or the process instructed to be executed by the user's input, and can gain auditory interest through the character. The user becomes more likely to feel familiar with the character, which in turn provides a strong motivation to use the interface device to instruct the processing device to execute a process. The character's voice is preferably a voice that is uttered in association with the character, and is particularly preferably a character's voice. In this case, it is particularly preferable that the input that the reception unit accepts is the voice input from the user. In this way, the user can more easily feel as if he or she is remotely controlling the processing device while communicating with the character through conversation.

(5) The processing device outputs the result of the executed process to a display unit on the processing device side, and the output unit of the interface device outputs the fact that the process has been executed by associating it with the character. It's good to do that.

In this way, the user can check the results of the processing executed by the processing device on the processing device side, and can confirm through the character that the processing has been executed on the interface device. The display section on the processing device side may be a display section included in the processing device, or may be a display section of an external display device connected to the processing device.
In addition to functioning as a device for remotely controlling a processing device, the interface device can also function as a device that has hobby-like or toy-like properties, such as being able to confirm through a character that processing has been executed.

(6) The processing device and the interface device may be connected via a wireless communication path whose communicable distance is a predetermined distance or less, or may be connected via a wired communication path.

In this way, by inputting an input to an interface device near the user, the user can cause a processing device also near the user to execute a process corresponding to the input. If the configuration is such that not only the interface device but also the processing device can accept input for instructing the execution of a process, the user can instruct the execution of a process by operating either the interface device or the processing device. When an input to the device instructs execution of a process, information corresponding to the input or process is output as information associated with the character, which can motivate the user to use the interface device. As a wireless communication channel whose communicable distance is less than or equal to a predetermined distance, it is particularly preferable to use a wireless LAN or short-range wireless communication. As the wireless LAN standard, for example, Wi-Fi (registered trademark) may be used. For example, Bluetooth (registered trademark) may be used as short-range wireless communication.

(7) The output unit may output the state of the processing device in association with the character.

In this way, the state of the processing device is output to the interface device in association with the character, so the user can recognize the state of the processing device through the character. For example, when a character is displayed, the user can recognize it more visually and visually than when the status of the processing device is simply output as a numerical value or the like. In particular, the output associated with the character may be an output in which the state of the processing device is personified using the character. An example of an output in which the state of the processing device is anthropomorphized using a character is an output using a method of changing the character's facial expression or character action, which will be described later. The output associated with the character may be an output in which an image indicating the state of the processing device is associated with the image of the character. Such output includes an output in which a character is displayed holding an object (for example, a board-shaped object) that displays the state of the processing device. The output associated with the character may be an output in which the voice of the character is associated with the voice indicating the state of the processing device. Such output includes an output that uses character voices to guide the status of the processing device.

(8) The output unit may output the processing device according to a resource state of the processing device.

In this way, the state of the resource of the processing device is output as the state of the processing device in association with the character, so the user can recognize the state of the resource through the character. For example, if the status of the character is output in accordance with the status of the resources of the processing device, the user can enjoy recognizing the status of the device. In this way, when a character is displayed on the interface device, for example, compared to simply outputting the resource status of the processing device in the form of characters, it is possible to recognize the character visually.

(9) The output unit may output a battery state as the resource state of the processing device.

In this way, the battery status is output as the status of the processing device and the resource status of the processing device in association with the character, so the user can recognize the battery status through the character. The state of the battery may be, for example, at least one of the remaining amount of the battery, charging or discharging.

(10) The output unit may output the state of the processing device based on a change in the facial expression of the character or an action of the character.

In this way, the user can recognize the state of the processing device through the character's facial expression and the character's motion, since it is output based on the character's facial expression and changes in the character's motion. For example, if the character's expression changes or the character's movement changes depending on the state of the processing device, the user can grasp the state of the processing device by checking the character's expression and character movement. can do.

(11) The reception unit may receive a message from the user, and the control unit may execute processing based on the message or a phrase included in the message.

In this way, by inputting a message into the interface device, the user can cause the processing device to execute a process based on the message or a phrase included in the message, and the user can instruct the processing device to execute the process. can be input in an intuitive manner.

(12) The reception unit may prompt the user to input a message again based on the message, and the control unit may execute processing based on the message and/or the message received again.

In this way, the user inputs the message into the interface device, is prompted to input the message again based on the input message, and can input the message again. By exchanging messages with the interface device, the user can execute desired processing from one or more messages. The interface device prompts the user to accept the message in order to further specify the desired process from the message received from the user. In this way, by exchanging messages multiple times like a game of catch, the user can feel as if he or she is having a pseudo conversation with the character on the interface device. Furthermore, by inputting a message multiple times by the user, it becomes possible to specify the process to be executed by the processing device step by step.

(13) The interface device may further include a voice input section, and the reception section may accept voice input from the user through the voice input section.

This allows the user to use voice when inputting to the interface device. For example, what the user says to the interface device is input to the interface device. Then, it is preferable that the character be outputted in association with the content of what the user has said to the interface device.

(14) The interface device may recognize a message from the input voice, and the control unit may execute processing based on the message or a phrase included in the message.

In this way, the user can execute the desired process on the processing device by including a message or a phrase in the message corresponding to the process the user wants to execute on the processing device when inputting information to the interface device. The interface device may be configured to recognize a message from the input voice. In this case, since the interface device recognizes the voice, the processing device does not need to perform a process of recognizing the voice.

(15) The output unit may change and output the character according to the voice input by the user.

In this way, when the user makes a sound to the interface device, the interface device can recognize a character that changes according to the sound. For example, when a user speaks to an interface device, the interface device changes its character in response to the conversation. Here, the change in the character may be, for example, changing the movement of the character, or may be, for example, changing the display of the character's facial expression, posture, and movement, or the voice emitted by the character.

(16) The reception unit transmits a command corresponding to the received input to the processing device, and the control unit receives the command and performs control to execute a process corresponding to the command. good.

In this way, a user can input a command to the interface device and have the processing device execute a process corresponding to the command. In a configuration in which the reception unit sends a command corresponding to the input to the processing device, and the processing device executes the processing corresponding to the command, it is sufficient for the processing device to be able to interpret the command received from the interface device, and it is sufficient for the processing device to be able to interpret the command received from the interface device. It is not necessary to have a configuration that accepts input from the user or interprets input from the user. Therefore, the application executed on the processing device is preferably an application that uses the resources of the processing device as little as possible. For example, it is sufficient to implement the minimum necessary functions in an application. For example, the size of an application executed on a processing device can be reduced, or an application executed on a processing device can be implemented by reducing the usage rate of the CPU of the processing device. or reduce memory at runtime.

(17) The command may correspond to an interface that utilizes a function of an operating system of the processing device or an application executed on the processing device.

In this way, the processing device may be configured to execute the functions of the operating system or execute the application based on the commands sent from the interface device. To execute the functions of the operating system, for example, when the operating system is Windows, it is preferable to use the Windows API.

(18) The control unit may execute an interface that utilizes functions of an operating system executed in the processing device or an application installed in the processing device in response to input from a user to the interface device.

In this way, by providing desired input to the interface device, the user can cause the processing device to execute an interface that utilizes the functions of the operating system in accordance with the user's input, or to execute an application in the processing device. You can

(19) The processing device may obtain input from a user to the interface device using a resident application.

In this way, the processing device can receive input from the user at the interface device by the application resident to obtain the input. Since the processing device has an application resident thereon, if a user inputs an input to the interface device at any time, the processing device can accept the input at any time and execute processing according to the input. Further, the application executed in the processing device only needs to have at least the function of interpreting commands from the interface device, and the capacity of the resident memory can be easily reduced.

(20) The resident application may receive commands from the interface device and obtain input from the user by interpreting the received commands.

In this case, the processing device may have a configuration in which a resident application is provided with a function of receiving and interpreting commands from the interface device. Since it is sufficient for an application residing in the processing device to be able to interpret commands received from the interface device, the application can minimize the use of the resources of the processing device.

(21) The output unit may output a state of the different device depending on a state of the device different from the processing device.

In this way, the user can recognize the state of a device other than the processing device in association with the character.

(22) The different device may be a device having a function different from that of the processing device.

In this way, the user can see that the different device is not just an information processing device, etc., but also a state related to a device that is different from the processing device and has a different function from the processing device, as well as the state associated with the character. It can be recognized by

(23) Preferably, the device having a function different from the processing device is a battery, and the output section changes and outputs the character in association with the state of the battery.

In this way, the user can recognize the state of the battery as a device that is different from the processing device and has a different function from the processing device in association with the character. Further, it is preferable that the character changes depending on the state of the battery. For example, the battery status, such as charging, completed charging, or insufficient remaining power, is associated with a character and output as a change in the character, allowing the user to recognize the battery status just by looking at the character. Furthermore, compared to simply displaying the remaining battery power, the user can enjoy visual and auditory enjoyment because the display is associated with the character's status.

(24) The output unit may output different characters between the processing device and the different device.

In this way, since the processing device and devices different from the processing device are displayed as different characters, the user can recognize the states of multiple devices at once. Furthermore, since different characters are output for different devices, the user can grasp the state of the device he or she wants to recognize just by looking at the characters. Furthermore, since the characters are associated with the device, depending on the type of character, the characters can be acquired in a state that is familiar to the user or in a state that is easy for the user to understand.

(25) The output unit may display a different character image or output a different sound for each character.

In this way, the user can visually recognize the status of a plurality of devices as images of different characters for each device. In addition, since each device is different, for example, the image that the user visually recognizes may be displayed as an image of a different character on a processing device and a device that is different from the processing device and has a different function from the processing device. will be done.

For example, the user can recognize the desired state of the device simply by visually recognizing the character without looking specifically at the interface device. Further, the different characters may be configured as characters that are related to, for example, a processing device and a device that is different from the processing device and has a different function from the processing device. For example, a character corresponding to a processing device may be displayed as an anthropomorphic character, and a character of a device different from the processing device may be displayed as a pet of the anthropomorphized character. With such a configuration, the user can recognize the relationship between the processing device and a device that is different from the processing device and has a different function from the processing device.

(26) The output unit may output the state in association with the state of a predetermined character.

In this way, the user can recognize the state of the processing device, the state of a device different from the processing device, or the state of the battery in association with the character. In particular, it is preferable that the state of the processing device, the state of a device different from the processing device, or the state of the battery be output in a form anthropomorphized into a character. In this way, by coming into contact with the character's state, the user can easily intuitively recognize the state of the processing device, the state of a device different from the processing device, or the state of the battery, and can also relate it to the character's state. You can also get some fun out of it. Aspects related to the character's state or anthropomorphized to the character include the character's expression, posture, character's voice, etc., depending on the state of the processing device, the state of a device different from the processing device, or the state of the battery. It is preferable to change the behavior of the character. In this way, the user can not only grasp the state of the processing device, the state of a device other than the processing device, or the state of the battery, but also have fun through the characters. It is particularly advantageous if the output is an image of the character, a sound of the character, or a combination thereof.

(27) The interface device can be further connected to a different processing device, and the interface device allows a different second processing device to execute processing on information corresponding to the processing acquired from the first processing device. It is a good idea to send it for use when doing so.

In this way, the user can instruct the first processing device to execute a process via the interface device, and can execute the process of the second processing device based on the executed result. A user can execute linked processing using processing results in different devices, such as a first processing device and a second processing device.

(28) The information according to the process may include information regarding a search result when the first processing device executes the search process.

In this way, the user can execute the search process on the first processing device via the interface device. The user can then use the execution results of this search process to execute new processing in the second processing device.

(29) The information regarding the search results may be location information.

In this way, the user may be able to execute a new process in the second processing device via the interface device using the location information as a search result obtained in the first processing device.

(30) The interface device may store specific information, and the control unit may acquire the specific information from the interface device and execute processing using the specific information.

In this way, when a user executes a process on the processing device, the user can execute the process using specific information stored in the interface device. With such a configuration, since specific information is stored in the interface device, for example, if the processing device to which the user interface device is connected is changed, the same specific information can be used.

(31) The specific information may be any one of personal information, setting information when executing the process, and information regarding login.

In this way, the user can store personal information, setting information for executing processing, and login information as specific information in the interface device. Since the user can store personal information etc. in the interface device rather than the processing device, the user can still use the personal information etc. stored in the interface device even if the processing device changes. Furthermore, it is preferable to store personal information and the like in an interface device different from the processing device instead of storing it in the processing device. For example, it is preferable that a user stores personal information and the like in an interface device that he/she personally uses, so that the personal information and the like are no longer stored in the processing device, thereby providing a system with high security.

(32) The processing device may further include a setting unit that associates and sets the input from the user and the process to be executed.

In this way, the user can set the content to be input by the user in accordance with the process to be executed by the processing device. In this way, the user can reconfigure the process to be executed in the processing device even if the content is not stored in the interface device. With such a configuration, for example, the user can register the process to be executed in the processing device with any desired content, and the user can customize the process to his or her liking.

(33) The setting unit may associate a message or a specific phrase input by the user with the process to be executed.

In this way, the user can set the process to be executed in the processing device in response to the input message or phrase. With such a configuration, the user can, for example, register the content of the message desired by the user or the process to be executed by the processing device in a specific phrase, allowing the user to customize the message to his or her liking.

(34) The specific phrase may be a phrase that has no relation to the process to be executed.

In this way, the user can arbitrarily set a phrase even if it is a phrase that is not related to the process executed by the processing device. Here, phrases that are not related to the process to be executed may be phrases such as "good morning," "goodbye," and "good night," which are not directly related to the process.

(35) The executed process may be an application executed on the processing device or an interface that calls a function of an operating system.

In this way, the user can set the application to be executed on the processing device and the interface for calling the function of the operating system on the processing device in correspondence with a specific phrase. This allows the user to set any process or program in accordance with the user's preferred phrase.

(36) The interface device stores the command and the message, and when the reception unit receives a message input from the user, the reception unit transmits a command corresponding to the message to the processing device, and The control unit preferably executes processing corresponding to the command.

In this way, when the interface device receives a message input from the user, it can send a command corresponding to the message to the processing device. The processing device may receive commands from the interface device and perform processing corresponding to the commands. The processing device only needs to receive commands from the interface device, and does not need to directly accept input messages from the user or interpret received messages.

(37) The setting unit may specify the process to be executed based on an operation input by a user in the processing device.

In this way, when setting a process to be executed on the processing device in response to an input from the user, the user can set the process based on the operation input on the processing device. The user only needs to operate the processing device and can easily set the processing.

(38) When the setting unit cannot specify a process to be executed based on the user's input, it is preferable that the setting unit sets a process specified based on the user's operation in the processing device in correspondence with the input.

In this way, when the user inputs information into the interface device but cannot specify a process to be executed based on the input contents, the user may perform an operation on the processing device to specify the process. The system may set the process specified by the user's operation in correspondence with the input made by the user to the interface device. When the user wants to register a new process, by performing an operation to execute the process on the processing device, the user can easily set the new process in association with the input performed by the user.

(39) Preferably, the output unit switches the output of the character depending on a history of the user's use of the interface device.

In this way, the output of the character will be switched depending on the history of the user's use of the interface device. For example, if the user uses the site many times, the display of the character may change or the character may evolve. Since the output of the character changes depending on the user's usage history, the user may be motivated to use the system more often, for example.

(40) The reception unit may change the type of input that can be accepted from the user depending on a history of the user's use of the interface device.

In this way, the type of input that the interface device can accept changes based on the user's history of using the interface device. For example, the more times a user uses the interface device, the more various inputs may be accepted. With such a configuration, the user may be motivated to use the system more often.

(41) Preferably, the control unit changes the process to be executed depending on a history of the user using the interface device.

In this way, the processing executed by the processing device changes based on the history of the user's use of the interface device. For example, it may be configured such that the more times a user uses the interface device, the more processes that can be executed by the processing device are increased. With such a configuration, the user may be motivated to use the system more often.

(42) The processing device has a character input mode that accepts character input from a user, the output unit outputs information based on the input in the character input mode, and the control unit It is preferable to execute the process instructed in response to the input in the character input mode.

In this way, the user can instruct execution of processing not only by operating the interface device but also by operating the processing device, and this can be reflected in the information output from the interface device.

(43) The character input mode preferably has a function of realizing conversational exchange between the user's character input and the characters output by the interface device.

In this way, by inputting characters into the processing device, the user can get the feeling of communicating in a conversational manner with the characters output by the interface device.

(44) The character input mode may display characters input by the user and characters acquired from the interface device in a conversational format.

In this way, by inputting characters, the user can feel as if he or she is having a chat-like conversation with the character output from the interface device. can.

(45) The interface device may transmit a signal for executing a character input mode to the processing device depending on the environment of the interface device.

In this way, depending on the environment of the interface device, the processing device is switched to a mode in which character input is possible, and the user can input characters in the processing device. For example, depending on the environment of the interface device, do you want the processing device to execute the processing device through remote control using the processing device, or do you want the processing device to execute the processing in the character input mode of the processing device? can be controlled.

(46) The environment of the interface device may be an environment based on a location where the interface device is installed.

In this way, based on the location where the interface device is installed, the processing device is switched to a mode in which character input is possible, and the user can input characters in the processing device. For example, depending on the location where the interface device is installed, do you want the processing device to execute the processing device by remote control using the processing device, or do you want the processing device to execute the processing in the character input mode of the processing device? You can control what happens.

(47) The location where the interface device is installed may be determined depending on the connection destination to which the interface device is connected.

In this way, the processing device is switched to a mode in which character input is possible based on the connection destination to which the interface device is connected, and the user can input characters in the processing device. The connection destination to which the interface device is connected may be determined based on, for example, information regarding a wireless LAN base station (eg, access point), such as the SSID.

(48) The environment of the interface device may be determined according to the volume level of sound acquired by the interface device.

In this way, depending on the volume level of the sound acquired by the interface device, the processing device switches to a mode in which character input is possible, and the user can input characters in the processing device. The volume of the sound acquired by the interface device may be determined based on, for example, the volume of the user's speaking voice. In this case, if the user's voice is low, it may be determined that character input is more suitable. Further, the volume of the sound acquired by the interface device may be determined based on, for example, environmental sound. For example, the interface device may determine that the environment is such that the user cannot speak when the environmental sounds are quiet. Further, for example, when the interface device determines that the user is in a public place or on a train based on the environmental sounds, it is preferable to determine that the user is in an environment where the user cannot make a sound.

(49) The interface device may include a housing having the output section, and the housing may have an appearance related to the character.

In this way, the user can recognize that information corresponding to the user's input or the process instructed to be executed by the input is output from the casing side that has an appearance related to the character. You can get more visual interest. The appearance associated with a character is particularly preferably an appearance that imitates that character. In this way, the user can feel as if he or she is requesting (for example, asking) the character to perform a process using the process, and can even use the interface to instruct the processing device to perform the process. Motivating. The appearance associated with a character may be an item used by the character, or may have an appearance that symbolizes the character.

(50) The interface device is an interface device that is connected to the processing device so as to be able to communicate with each other, and includes a reception unit that receives the input, and a reception unit that receives the input from the user or responds to the input from the user to the interface device. It is preferable to include an output unit that outputs information based on the processing by associating it with a predetermined character.

In this way, by inputting input to the interface device external to the processing device, the user can cause the processing device, which is a device different from the interface device, to execute processing according to the user's input, and also Alternatively, information corresponding to the executed process can be recognized through the interface device by associating it with the character.

(51) The program may be a program for causing a computer to implement the functions of the control unit of the system described in any one of (1) to (49) above.

The inventions shown in (1) to (49) above can be combined arbitrarily. For example, at least a part of the structure of at least one invention described in (2) and subsequent ones may be added to all or part of the structure of the invention shown in (1). In particular, it is preferable to create an invention in which at least a part of the structure of at least one invention after (2) is added to the invention shown in (1). Further, arbitrary configurations may be extracted from the inventions shown in (1) to (49) and the extracted configurations may be combined. The applicant of this application intends to acquire rights to inventions containing these structures. Furthermore, even if there is a description of "in the case of" or "at the time of", the description is not intended to be limited to those cases or times. These are examples of better configurations, and we intend to acquire rights to these cases and other configurations as well.

Furthermore, the sections described in order are not limited to this order. It also discloses a configuration in which some parts have been deleted or the order has been changed, and we have the intention to acquire the rights.

According to the system or the like of the present invention, it is a system or the like in which the interface of the processing device is realized by a different device, and it is possible to provide a technology different from the conventional technology.

Note that the effects of the invention of the present application are not limited to these, and effects obtained from the parts of the configuration disclosed in the present specification, drawings, etc. are also disclosed, and the configurations that provide the effects are also disclosed by divisional applications, amendments, etc. Have the intention to acquire the rights. For example, in this specification, passages such as "can be done," "is possible," etc. are descriptions that clearly indicate the effect to be achieved, and even if there is no description such as "can be done," or "it is possible," the effect can be obtained. There is a part shown. Further, even without such a description, there are effects that can be understood from the configuration.

FIG. 1 is a diagram for explaining an overview of a system in a first embodiment. FIG. 2 is a diagram for explaining the hardware configuration in the first embodiment. FIG. 2 is a diagram for explaining the configuration of software in the first embodiment. FIG. 2 is a diagram for explaining the configuration of software in the first embodiment. FIG. 2 is a diagram for explaining (a) an example of the data structure of a command table and (b) an example of the data structure of a setting table in the first embodiment; FIG. FIG. 3 is a diagram for explaining main processing in the first embodiment. FIG. 3 is a diagram for explaining recognition processing in the first embodiment. FIG. 3 is a diagram for explaining an example of operation in the first embodiment. FIG. 3 is a diagram for explaining an example of operation in the first embodiment. FIG. 3 is a diagram for explaining an example of operation in the first embodiment. FIG. 7 is a diagram for explaining a hardware configuration in a third embodiment. FIG. 7 is a diagram for explaining main processing in a third embodiment. FIG. 7 is a diagram for explaining an example of operation in a third embodiment. FIG. 7 is a diagram for explaining an example of operation in a fourth embodiment. FIG. 7 is a diagram for explaining an example of operation in a fourth embodiment. It is a figure for explaining main processing in a 5th embodiment. It is a figure for explaining the example of operation in a 5th embodiment. It is a figure for explaining the example of operation in a 5th embodiment. It is a figure for explaining the example of operation in a 6th embodiment.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In addition, the embodiment shown below is one embodiment which provided this invention, and the content of this invention is not limited and interpreted based on the following description.

2. Description of the Related Art Generally, an assistant function for assisting a user's operations is known in a processing device such as a personal computer (PC). Known examples of such assistant functions include Microsoft's Cortana and Apple's Siri.

Although these assistant functions seem convenient, they only provide assistance, and users find them boring or do not feel the need to perform them, so they lack motivation to use them.

Therefore, it would be interesting if a character, such as an anime character or mascot character, could perform the assistant function in the form of an agent, and the user would be more motivated to use it. In particular, if the user uses his/her favorite anime, game characters, favorite mascot characters, or corporate characters, this can motivate the user to actively use them.

Here, in order to make the character move on the terminal device, it is conceivable to execute an application using a game engine such as Unity, for example. However, in order to execute an application for making a character move on a processing device, a certain amount of resources are required. Furthermore, execution on a processing device creates a restriction that the interface is limited to that of the processing device.

An embodiment of the system of this embodiment, which aims to solve such a problem, will be described below. Note that the following embodiment is an example for realizing the present invention. Therefore, the content of the present invention is not limited to the form of the following embodiments.

[1. First embodiment]
[1.1 Entire system]
FIG. 1 is a diagram illustrating an overview of the system 1. As shown in FIG. System 1 includes an interface device 10 and a processing device 20.

The interface device 10 is a device that can communicate with the processing device 20. The interface device 10 includes, for example, a display section 140 as a display device. For example, a character is displayed on the display section 140. The character represents, for example, a person, and can show facial expressions of joy, anger, sorrow, and happiness.

The processing device 20 is a device capable of executing various types of processing. For example, the processing device 20 is preferably a personal computer (personal computer/PC) capable of executing general-purpose processing. Further, the processing device 20 is not limited to a personal computer capable of executing general-purpose processing, but may also be a processing device such as a tablet, or a device capable of executing processing for a specific purpose such as a car navigation system or a drive recorder.

The processing device 20 and the interface device 10 may be connected via a wireless communication path whose communicable distance is a predetermined distance or less, or may be connected via a wired communication path. If the interface device 10 and the processing device 20 are connected via a wireless communication path whose communicable distance is a predetermined distance or less, for example, the interface device 10 and the processing device 20 may use short-range wireless communication such as BLE (Bluetooth Low Energy). good. In the case of a wired communication path, cables of USB, HDMI (registered trademark) (High-Definition Multimedia Interface), or other standards may be used. By inputting information to the interface device 10 located near the user, the user can cause the processing device 20 also located near the user to execute processing according to the input. Since not only the interface device 10 but also the processing device 20 is configured to accept input for instructing the execution of a process, the user can instruct the execution of the process by operating either the interface device 10 or the processing device 20. However, when the execution of a process is instructed by input to the interface device 10, information corresponding to the input or process is output as information associated with the character, so that the user can use the interface device 10. It can be used as motivation.

A user prepares an interface device 10, which is an external device to the processing device 20. A lightweight application for communicating with the interface device 10 may be prepared and installed on the processing device 20. With such a configuration, the processing device 20 can separate many of the required resources to the interface device 10, which is an external device.

Further, the interface device 10 is capable of inputting instructions from a user, for example. In this embodiment, a message specifying the content of the instruction can be input as the instruction from the user. The interface device 10 generates a command corresponding to a message input by the user, converts it into a format that can be interpreted by an application running on the processing device 20, and transmits the command. Note that the interface device 10 may generate a command based on a phrase included in a message. Note that it is preferable that messages and phrases have at least a linguistic meaning that can be understood by humans.

The processing device 20 is configured to use an API (Application Programming Interface) of the operating system of the processing device 20 to execute a target process when the application receives a command. The application executed on the processing device 20 is particularly preferably executed as a resident application. The resident application may be an application that continues to operate while the processing device 20 is powered on, or an application that is started when the OS is started, even if the user does not explicitly instruct it to start. It is preferable that the application is executed in the processing device 20.

In this way, when the user inputs a message to the interface device 10, the processing device 20 executes processing according to the message. That is, the interface device 10 can instruct the processing device 20 to directly execute an API or execute a command.

Note that this interface device 10 may be, for example, LVR-01 sold by Jupiter Co., Ltd., the applicant. In this product, Rei Kirishima, a character used by Jupiter Co., Ltd., is displayed as a virtual figure. In the drawings corresponding to the embodiments below, this character is used as an example, but the invention is also applicable to other characters.

In the system 1, by inputting input to an interface device 10 located outside the processing device 20, the user causes the processing device 20, which is a device different from the interface device 10, to execute processing according to the user's input. , information corresponding to the input or executed processing can be recognized through the interface device by associating it with the character.

The input that can be accepted by the interface device 10 is particularly preferably an input that can be intuitively performed by the user, such as a voice uttered by the user or a gesture movement shown by the user. In this way, the user's input burden on the interface device 10 can be reduced. Note that the input that can be accepted by the interface device 10 may be input other than these, for example, input that can be accepted using an input means such as a physical key or a touch sensor. For example, the interface device 10 may accept input of characters composing a message.

The processing executed by the processing device 20 is preferably a process that can be executed by the processing device 20, and particularly preferably a process executed by a function of application software installed in the processing device 20. The application software may be an OS or a program running on the OS. In this way, the user can execute a process that can be executed by the functions of the OS installed in the processing device or the application running on the OS by inputting it to the external interface device. Since the user does not have to input instructions to the processing device about the OS, an application running on the OS, or a process to be executed by the OS, the input burden on the user can be reduced.

Further, the interface device 10 preferably outputs information that can be recognized by the user in association with the character, and particularly preferably outputs images or sounds. This makes it easier for the user to recognize information that corresponds to his or her own input or processing. The information corresponding to the process input or executed may be information indicating that the input has been received or an instruction to execute the process, but it is particularly preferable to use information that allows the user to recognize the result of the process. In this way, the user can recognize the results of the processing performed by the processing device 20 not through the processing device 20 but through the interface device 10.

[1.2 Hardware configuration]
The hardware configuration of each device will be described below with reference to FIG. 2. FIG. 2 is a diagram showing an example of the hardware configuration of the interface device 10 and the processing device 20.

[1.2.1 Interface device]
The interface device 10 includes a control section 100, a storage 110, a ROM 120, and a RAM 130 as storage devices, a display section 140, an input section 150, an audio input section 160, an audio output section 162, and a communication section 170. There is.

The control unit 100 is a functional unit for controlling the entire interface device 10. The control unit 100 realizes various functions by reading and executing various programs stored in the storage 110 and the ROM 120, and implements various functions by reading and executing various programs stored in the storage 110 and the ROM 120. (System on a Chip)).

The storage 110 is a nonvolatile storage device that can store programs and data. For example, it may be configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Further, the storage 110 may be an optical disc such as a DVD/BD, or may be configured as a USB memory. Further, the storage 110 may be a storage area on a cloud, for example.

The ROM 120 is a nonvolatile memory that can retain programs and data even when the power is turned off.

The RAM 130 is a main memory mainly used by the control unit 100 when executing processing. The RAM 130 is a rewritable memory that temporarily holds programs read from the storage 110 or the ROM 120 and data including execution results.

The display unit 140 is a display device that can display various information and execution screens. The display unit 140 may be, for example, a display device such as a liquid crystal display (LCD), an organic EL (electro luminescence) display, or an electrophoretic display. Furthermore, the display unit 140 includes an interface to which a display device can be connected. For example, it may be configured with an external display device connected via HDMI (High-Definition Multimedia Interface), DVI (Digital Visual Interface), or Display Port. Note that the display unit 140 on the interface device 10 side may be a display unit built into the interface device 10 or may be a display unit externally attached to the interface device 10.

The input unit 150 is an input device (an example of an input means) that allows a user to input information. The input unit 150 accepts input that is different from the voice input that the voice input unit 160 accepts. Input unit 150 may include, for example, a non-contact sensor that accepts non-contact operations. Non-contact operations include gesture movements. As the non-contact sensor, it is preferable to use what is called an air touch sensor, but a camera or various sensors for recognizing gesture movements may also be used. The input unit 150 may include an input device such as a physical key or a touch sensor. The input unit 150 may be configured to be integrated with the display unit 140 and accept input from the user, and for example, may accept input regarding an image displayed on the display unit 140. Furthermore, the input unit 150 preferably includes an interface to which an external input device can be connected. For example, it may be configured with an input device such as a keyboard or mouse connected via USB (Universal Serial Bus) or wirelessly.

The audio input unit 160 receives sound (including voice) input, converts the sound into sound data, and outputs the sound data. The audio input section 160 mainly includes a microphone, but may also include an external microphone connected via an audio interface, USB, or wireless interface.

The audio output unit 162 outputs sound (including audio) from the sound data. Although it is mainly composed of speakers, it may also be composed of external speakers, headphones, etc. connected via an audio interface, USB, or wireless interface. Furthermore, it is assumed that the sound includes not only the sound emitted by humans but also the sound output by devices (interface device 10, processing device 20).

The communication unit 170 is a communication interface that connects with other devices. The communication unit 170 may be, for example, a network interface that can provide wired connection or wireless connection. In this embodiment, short-range wireless communication (for example, Bluetooth (registered trademark)) that can be connected to the processing device 20 at a short distance is preferable. Alternatively, a network adapter that can connect to a wireless LAN according to standards such as IEEE802.11a/b/g/n may be used. Further, the communication unit 170 may be connected to the processing device 20 by wire via an interface such as a USB.

[1.2.2 Processing device]
The processing device 20 includes a control section 200, a storage 210, a ROM 220, a RAM 230, a display section 240, an operation section 250, and a communication section 270. Among these configurations, if there is a configuration with the same name in the above-mentioned interface device 10, the basic configuration is the same as that configuration. The display unit 240 on the processing device 20 side may be a display unit built into the processing device 20 or may be a display unit externally attached to the processing device 20. The operation unit 250 receives operation input from the user. The operation unit 250 may have an operation means different from the processing device 20, such as a keyboard or a numeric keypad, for example, an operation means richer than the processing device 20. The operation unit 250 may have a configuration that accepts the same operations that the processing device 20 can accept, such as operations by voice input.

[1.2.3 Other configurations]
Note that the configuration in FIG. 2 is an example of a typical configuration of the interface device 10 and the processing device 20, and is not limited to the configuration shown in FIG. 2. As long as it is obvious to those skilled in the art, other configurations may be used as necessary, or the configuration shown in FIG. 2 may not be provided. For example, the communication unit 170 and the communication unit 270 may include a plurality of communication units. For example, it may further include a communication interface for connecting to a mobile communication network (for example, a communication module capable of communicating with the mobile communication network using 4G/5G, etc.).

[1.3 Software configuration]
Next, the software configuration in the system 1 will be explained with reference to FIGS. 3 and 4. With the following configuration, an assistant application that is generally implemented in the operating system of the processing device 20 can be characterized, and resources can be separated from the operating system in the processing device 20.

[1.3.1 Interface device]
The interface device 10 has an application 1000 installed therein. When the control unit 100 of the interface device 10 executes the application 1000, each function is realized by the application 1000. Further, in the interface device 10, the application 1000 may be executed on an operating system (OS) that is executed on the interface device 10.

The interface device 10 also implements a voice recognition section 1100, a conversation control section 1300, a character control section 1400, and an information acquisition section 1500. The interface device 10 also stores a command table 1200.

Furthermore, by executing the application 1000, the interface device 10 implements a message interpretation section 1010, a command generation section 1020, a command transmission section 1030, and a response information reception section 1040.

Note that each function may be realized by the control unit 100 reading a program stored in the ROM 120 or the storage 110, or may be realized by using a service provided on the network. You may.

Further, the voice recognition section 1100, the conversation control section 1300, the character control section 1400, and the information acquisition section 1500 may be executed by the application 1000, or may implement functions that the interface device 10 has in advance. Further, the functions realized by the application 1000 may be realized by a program other than the application 1000, if necessary.

The voice recognition unit 1100 recognizes the voice uttered by the user as a message. The speech recognition unit 1100 may perform speech recognition, for example, by acquiring a sound emitted by a user as sound data and comparing it with a predetermined sound pattern. Further, the speech recognition unit 1100 may perform speech recognition using machine learning or deep learning. Speech recognition may be a process of converting sound data into a character string that is text data.

The conversation control unit 1300 performs a conversation by responding in accordance with the message recognized by the voice recognition unit 1100. The conversation control unit 1300 may output a response to the message received from the user as a sound from the audio output unit 162, for example. Furthermore, the conversation control unit 1300 may perform a display that changes the character's facial expressions and actions via the character control unit 1400. For example, when the conversation control unit 1300 responds by outputting a voice in response to a message issued by the user, the display of the character may be changed. Specifically, the character control unit 1400 may move the character's mouth in accordance with the conversation to create the illusion that the character is actually speaking. Furthermore, the conversation control unit 1300 may change the facial expression of the character depending on the content of the conversation. The character's facial expression may be, for example, a facial expression expressing joy, anger, sorrow, or happiness.

The conversation control unit 1300 outputs audio corresponding to the user's message and changes the display mode of the character, so that the user can enjoy the conversation as if he were having a conversation with the character displayed on the interface device 10. can.

Furthermore, the conversation control unit 1300 may output a response to a message input by the user in text. In this case, conversation control section 1300 may display characters via display section 140. Further, the conversation control unit 1300 may respond to a conversation with the user. For example, when the conversation control unit 1300 receives the voice uttered by the user, it may output a voice saying "I understand" as a response. Further, when the conversation control unit 1300 cannot recognize the voice uttered by the user, it may output a voice such as "I didn't understand" or "Please repeat again." Additionally, the conversation control unit 1300 may prompt the user to input multiple messages.

The character control unit 1400 controls the output of the character displayed on the display unit 140 of the interface device 10. For example, the character control unit 1400 may control the movement of the character display as one of the character outputs. The movement of the character may be controlled, for example, by controlling the movement or facial expression of the character. The character's facial expressions may include facial expressions such as the character's expressions of joy, anger, sadness, and surprise. Further, the character's actions may include actions such as the character winking, making a peace sign, running, and going to bed. Further, the character's motion may include the character's posture. For example, the character's motion may include the character's sitting, standing, and sleeping postures.

Further, the character's movement may include not only visual movement but also auditory movement. For example, the character control unit 1400 may cause the character to make a sound. For example, voices such as "Good morning," "Thank you," "Thank you," and "I'm sorry" may be uttered. The voices of these characters may be output as the voice of a character in general, such as a voice actor who is in charge of the character in an anime, for example. Further, the user may associate arbitrary voices.

Further, the character may be a character that imitates a human in appearance, or may be an animal, a plant, or a virtual character. Further, the characters may be anthropomorphized animals, plants, or other objects.

The character may be, for example, a personification of the processing device 20. For example, the processing device 20 may be personified with different characters such as a computer, a mobile battery, a car navigation device, etc. When the character changes according to the processing device 20, the user can identify the processing device 20 to which the interface device 10 is connected just by looking at the character.

The information acquisition unit 1500 acquires information regarding the processing device 20. For example, via the OS 2200 of the processing device 20, information set in the processing device 20, information regarding the state of the processing device 20, information regarding the execution results of programs executed in the processing device 20, etc. can be acquired.

The information set in the processing device 20 includes, for example, an IP address, an OS type, information regarding installed applications, a machine name, and the like. Further, the information regarding the state of the processing device 20 includes, for example, information such as the free capacity of the storage of the processing device 20, the remaining capacity of the battery, and the amount of memory used. Further, information regarding the execution result of the program executed by the processing device 20 includes information such as whether the program was executed correctly or could not be executed due to an error.

Note that in this embodiment, the response information receiving unit 1040 receives the results executed by the application 2000 of the processing device 20, but the information acquiring unit 1500 may also receive the results.

Next, the message interpretation section 1010, command generation section 1020, command transmission section 1030, and response information reception section 1040 that are realized when the application 1000 is executed will be explained.

Message interpreter 1010 interprets messages received from users. Here, a message is a message received from a user, such as a voice message, a message received by inputting a switch, etc., a message from an input device such as a keyboard or a mouse, a message by text input, a message by the user's gesture, etc. including. The message in this embodiment will be explained using an example of a message input by a user using voice and obtained by recognizing the voice. The message interpretation unit 1010 interprets what kind of message the user has uttered from the voice recognized by the voice recognition unit 2100.

The command generation unit 1020 generates a command corresponding to a user's message. Here, the command is an instruction corresponding to a process to be executed by the processing device 20, and is an instruction that can uniquely specify the process that the processing device 20 is instructed to execute. The command may be set in advance by the system or may be set by the user. For example, the command corresponding to the message input by the user may be in a form that can be interpreted by the application 2000 of the processing device 20. For example, as will be described later, the application 2000 of the processing device 20 only needs to be able to interpret the received command and execute the desired process using, for example, an API of the OS. Further, the command corresponding to the message may correspond to a specific phrase included in the message.

The command generation unit 1020 stores commands to be sent from the interface device 10 to the processing device 20. The command generation unit 1020 may refer to the command table 1200 and generate a command corresponding to the message.

FIG. 5(a) is an example of the command table 1200. The command table 1200 stores commands and message contents in association with each other. Command table 1200 may be generated based on setting table 2300 set in processing device 20.

For example, as described in the command table 1200 in FIG. 5A, the command "WS OPEN" is associated with the message content "display list of spreadsheets." Here, the command "WS" indicates executing a spreadsheet application, and "OPEN" indicates displaying a list.

The message content stores the content of the message, and in this embodiment, the words uttered by the user as voice are stored as an example. The words stored in the message content may be the voice uttered by the user as is, or may be a specific phrase included in the message.

Further, as described in the command table 1200 in FIG. 5A, commands may be generated in correspondence with a plurality of phrases as message contents. For example, the command "WS YS" is stored for a message containing the phrases "yesterday" and "spreadsheet file." Here, for example, the command "YS" may be a command indicating "yesterday". By setting the content of the message as a phrase, the user is not affected by differences in detailed expressions, and appropriate commands can be generated. For example, if the phrases ``yesterday'' and ``spreadsheet file'' only need to be included, it will be processed appropriately regardless of whether the user utters ``I want to see yesterday's spreadsheet file'' or ``yesterday's spreadsheet file''. Ru.

Additionally, the command may include parameters. For example, the command "DOC %1" is associated with the message content "file of text %1". Here, the command "DOC" is, for example, a command for starting a text editing application (for example, a word processing application). Moreover, "%1" indicates an argument that is a parameter, as an example. For example, when the user utters "document nippou file", the command generation unit 1020 may generate the command "DOC nippou".

The command transmitter 1030 transmits the command generated by the command generator 1020 to the processing device 20. For example, the command transmitter 1030 may transmit a command to the processing device 20 via the communication unit 170 and the communication unit 270.

The response information receiving unit 1040 receives response information from the processing device 20. Here, the response information is information transmitted by the processing device 20, and may include information indicating the result of processing executed by the application 2000. The response information may include, for example, an execution result when the application 2000 executes a process corresponding to a command.

In the interface device 10, information corresponding to the user's input or the processing executed by the input is output in association with a predetermined character, so that the user can easily feel familiarity with the character, and as a result, the interface This provides a strong motivation to use the device 10 to instruct the processing device 20 to execute processing. The input that can be accepted by the interface device 10 may be a voice input from a user. In this way, the user can feel as if he or she is having a conversation with the character, and the user can more easily feel that he or she is communicating with the character.

Furthermore, since the interface device 10 outputs the character, the user can visually recognize the image of the character that changes according to the user's own input or the process instructed to be executed by the input. Therefore, the user can gain visual interest through the character.

Further, the user can listen to the character's voice, which changes depending on the user's own input or the process instructed to be executed by the user, from the interface device 10. The user can enjoy auditory fun through the characters.

In this way, by using the interface device 10, it is possible to control the processing device 20 via the character. Since the interface device 10 outputs the output based on the character, the user can feel visually, audibly, etc., and become more familiar with the character. This provides a strong motivation to instruct the device to perform processing. The character's voice is preferably a voice that is uttered in association with the character, and is particularly preferably a character's voice.

[1.3.2 Processing device]
The processing device 20 has an OS 2200 installed therein, and an application 2000 installed therein. When the control unit 200 of the processing device 20 executes the application 2000, each function is realized by the application 2000. Furthermore, the processing device 20 may execute the application 2000 on the OS 2200 that is executed by the processing device 20 .

Furthermore, the processing device 20 implements a command receiving section 2010, a command interpreting section 2020, a command executing section 2030, and a response information processing section 2040. The processing device 20 also stores a setting table 2300.

Note that each function may be realized by the control unit 200 reading a program stored in the ROM 220 or the storage 210, or may be realized by using a service provided on the network. You may.

The functions realized by the application 2000 may be realized by a program other than the application 2000, if necessary.

OS2200 is a general operating system. The OS 2200 is any OS such as Windows (registered trademark), Mac OS (registered trademark), or Chrome OS (registered trademark).

The command receiving unit 2010 receives commands from the command transmitting unit 1030 of the interface device 10. Command receiving section 2010 transmits the received command to command interpreting section 2020.

The command interpreter 2020 interprets the received command and converts it into processing content that can be executed by the processing device 20. Here, the command interpreter 2020 refers to the setting table 2300 and interprets the command.

The processing content interpreted by the command interpretation unit 2020 is executed by the command execution unit 2030. Specifically, the control unit 200 executes a program based on the processing content sent from the command interpretation unit. Thereby, the processing content corresponding to the content of the message received by the interface device 10 from the user is executed in the processing device 20.

The setting table 2300 is a table for setting commands, message contents, and processing contents executed by the processing device 20. The setting table 2300 may be set in advance or may be set by the user.

An example of the setting table 2300 is shown in FIG. 5(b). The setting table 2300 stores commands, message contents, and processing contents in association with each other. For example, upon receiving the command “WS OPEN” from the interface device 10, the command interpretation unit 2020 transmits “Worksheet.exe” to the command execution unit 2030. The command execution unit 2030 starts the spreadsheet application by executing the processing content "Worksheet.exe".

As the processing content, a program to be executed may be stored, or an operating system interface may be stored. For example, an API (Application Programming Interface) that can be called in Windows may be stored. Further, the processing content may be stored as a Web API.

The command interpreter 2020 may generate processing details from the received command by referring to the setting table 2300. For example, the command "YS" is a command that indicates yesterday. Therefore, upon receiving the command "WS YS", the command interpreting unit 2020 may generate the processing content by replacing the command with the date of the day before the date of receiving the command. The setting table 2300 stores “Worksheet.exe [Today()-1]” as the processing content. "Today()" is a function that returns today's date, and the processing content described as "[Today()-1]" may be replaced with the day obtained by subtracting 1 from today's date (that is, the previous day).

Furthermore, when the command includes an argument, the command interpreter 2020 may replace the processing content with the argument. For example, "DOC %1" includes the argument specification "%1". Therefore, the command interpretation unit 2020 may replace the character string specified by the argument “%1” included in the command with the argument “%1” of the processing content “Doc.exe %1”.

The response information processing unit 2040 transmits response information based on the result of the command execution unit 2030 to the interface device 10 as necessary.

FIG. 4 shows a schematic diagram of the execution format in the processing device 20. The processing device 20 is running an OS 2200 and can run one or more resident applications 2500. In this embodiment, the application 2000 described above may be activated.

Furthermore, the processing device 20 may execute one or more execution applications that are executed in response to a startup operation from the user. For example, in FIG. 4, a spreadsheet application 2610, a text editing application 2620, an email application 2630, and a browser application 2640 may be installed as an example of the execution application 2600, and these applications can be started at any timing. can.

In general, it is preferable that the resident application 2500 has a smaller executable file size and a smaller amount of memory required during execution than the executable application 2600. The smaller the size of the resident application 2500, the smaller the load on the processing device 20.

Furthermore, the processing device 20 can control the startup and execution of each application via the OS 2200. Further, the OS 2200 may communicate with applications using an API as an interface. For example, the application 2000 can call and execute the necessary startup of the OS 2200 via the API 2210. The application 2000 may execute the execution application 2600 or obtain the execution result via the API 2210. Note that if the application 2000 can use the interface of the execution application 2600, the application 2000 may execute the execution application 2600 or obtain the execution result by directly controlling the execution application 2600.

Further, the OS 2200 may obtain information from the controller 2510 via the API 2210. The controller 2510 may obtain information about the hardware of the processing device 20, or may obtain information from an external device connected to the processing device 20, for example.

In this way, by separating the configuration related to the assistant function using characters from the OS 2200 into the interface device 10, which is an external device, the processing device 20 does not need to allocate resources to the assistant function using characters.

Further, the processing device 20 may install an application 2000 that simply executes processing of receiving commands from the interface device 10, which is an external device, interpreting the commands, and passing the commands to the API. Since the application 2000 does not need to perform processing such as character display or conversation control, it can be implemented with a small program size. Further, since the application 2000 has a small program size, the amount of resident memory can be kept small.

Further, the results of the processing executed by the processing device 20 can be confirmed through the processing device 20, and the fact that the processing has been executed in the interface device 10 can be confirmed through the character. The interface device 10 functions as a device that remotely controls the processing device 20, and can also function as a device that has hobby-like or toy-like characteristics, such as being able to confirm the execution of processing through a character.

[1.4 Process flow]
[1.4.1 Overall processing]
The flow of processing in this embodiment will be explained with reference to FIG. 6. FIG. 6 is a diagram illustrating the processing of the interface device 10 and the processing of the processing device 20.

First, the interface device 10 executes recognition processing (step S102). When the interface device 10 executes the recognition process, it generates a command according to the message received from the user. Then, the interface device 10 transmits the generated command to the processing device 20 (step S104).

Upon receiving the command (step S152), the processing device 20 executes command execution processing (step S154). That is, the command receiving unit 2010 receives commands from the interface device 10. The command interpretation unit 2020 interprets the contents of the process corresponding to the received command with reference to the setting table 2300. The command execution unit 2030 executes processing in the processing device 20 based on the interpreted content of the processing.

As a result, the processing device 20 executes processing corresponding to the message received by the interface device 10.

Further, when the processing device 20 needs to return the result of executing the command as a response to the interface device 10 (step S156; Yes), it transmits response information to the interface device 10 (step S158).

When the interface device 10 receives the response information from the processing device 20 (step S110), it executes response processing (step S112). By executing the response process, the interface device 10 can, for example, control the display of a character or output audio based on the information included in the response information.

In this way, according to the process shown in FIG. 6, the user can execute the necessary process on the processing device 20 by inputting a message to the interface device 10. Furthermore, the user can also check the response from the processing device 20 on the interface device 10.

[1.4.2 Recognition processing]
An example of the recognition process described in step S102 in FIG. 6 will be described with reference to FIG. 7. The control unit 100 receives voice input from the user (step S1002).

Subsequently, the control unit 100 recognizes the voice (step S1004). In step S1004, for example, the voice recognition unit 1100 recognizes the voice message and converts it into a character string (that is, text data) message. Further, the converted message is output to conversation control section 1300. The conversation control unit 1300 conducts a conversation with the user based on the input message. The conversation control unit 1300 may react depending on the conversation with the user. For example, the conversation control unit 1300 can output greetings, replies (for example, yes/no), and other questions in voice via the voice output unit 162. Furthermore, the conversation control section 1300 may change the display mode of the character being displayed via the character control section 1400 at this time. For example, the character control unit 1400 may make the character smile when the conversation control unit 1300 gives a positive reply.

By having the conversation control unit 1300 converse with the user in this way, the user can enjoy the conversation as if he were talking to a person through the character. Also, at this time, by changing the display mode of the character (for example, the character's facial expression, the character's body movement, posture, etc.), the user can visually improve the character's appearance as the character changes according to the conversation. It becomes possible to enjoy it.

Here, the control unit 100 interprets whether the message includes a specific phrase for executing processing in the processing device 20 (step S1006). When part or all of the message received from the user has a specific phrase to be processed by the processing device 20 (step S1008; Yes), the control unit 100 generates a command corresponding to the message or phrase. (Step S1010). Specifically, the command generation unit 1020 refers to the command table 1200 and generates a command.

[1.5 Operation example]
For example, a user speaks words including a specific phrase to the interface device 10. The interface device 10 acquires words uttered by the user (that is, spoken words). For example, upon recognizing the message "Good morning", the interface device 10 sends a command corresponding to the message to the processing device 20. The processing device 20 executes processing such as "starting a mailer", "starting a mailer and receiving mail", and "starting a browser" in response to a command, for example. Note that the message or phrase included in this utterance does not necessarily directly indicate the specific content of the instruction. That is, the message or phrase included in this utterance may be relevant to the application executed in the processing device 20. The related phrase is, for example, the phrase "receive mail" with respect to "mailer" and "mail sending/receiving application". In addition, unrelated phrases are, for example, phrases such as "good morning,""goodnight," and "goodbye."

For example, the system 1 can refer to the command table 1200, the setting table 2300, etc. in response to a message or phrase, and execute the processing content set by the user on the processing device 20. Note that the setting of the relationship between a message or phrase by the user (that is, a recognized phrase recognized by the interface device 10 or an uttered phrase uttered by the user) and a command or processing content will be described in another embodiment.

[1.5.1 First operation example]
A first operation example of this embodiment will be described with reference to FIG. 8. The interface device 10 displays a character in the display area W10 in a display mode V102.

Then, the user U1 utters a voice saying, "Search for the spreadsheet file you were looking at yesterday" (C102). The audio (C102) is input to the interface device 10.

The interface device 10 recognizes the input voice, recognizes a phrase for operating the processing device 20 from the recognized message (C102), and generates a corresponding command. At this time, for example, the processing device 20 may return to the user U1 that the message has been received. For example, the interface device 10 may output a voice saying "Okay" (C104). The audio (C104) output by the interface device 10 can be heard by the user U1. Furthermore, the interface device 10 displays a character in the display area W10 as a display mode V104. Here, in the display mode V104, the character's facial expression is displayed as a smiling face. Furthermore, in display mode V104, the character is making a peace sign as an action.

The user U1 can intuitively understand from the voice (C104) from the interface device 10 and the display mode V104 of the character that the process corresponding to the message is being executed correctly. Further, the user U1 can perform desired processing on the processing device 20 while conversing with the character. Furthermore, since the interface device 10 of this embodiment can have a simulated conversation with a character, it can provide a highly entertaining interface.

The interface device 10 transmits a command to the processing device 20 (M102). The interface device 10 generates a command “WS YS” that can be interpreted by the processing device 20 and sends it to the processing device 20.

The processing device 20 converts the received command into processing content (here, Worksheet.exe 20220215). Then, the processing device 20 executes the process (M104). By executing the process, the processing device 20 displays a file list screen of the spreadsheet application in, for example, the display area W12 (area R102). Area R102 displays a list of files with dates specified in the process.

[1.5.2 Second operation example]
A second operation example of this embodiment will be described with reference to FIG. 9. The second operation example is an operation example in which the user continues a plurality of conversations with the interface device 10.

First, the user U1 utters a voice saying "Open the text file" (C122). The audio (C122) is input to the interface device. In the interface device 10, a character is displayed in a display mode V122 in a display area W12.

For example, the message "Open text file" can have as an argument information (for example, file name, etc.) that more specifically specifies the text file to be opened. Therefore, the interface device 10 issues a voice to the user U1 asking, "What file do you want to open?" (C124). In this way, the interface device 10 may prompt the user to enter a message again.

Further, the interface device 10 may change the display mode of the character to display mode V124. For example, since the interface device 10 cannot specifically specify the file to be opened, the character may have an angry expression or a troubled expression. In addition to facial expressions, the interface device 10 may display actions indicating trouble, such as the actions and postures of the character.

Since the user U1 has been asked by the interface device 10, "What file do you want to open?", the user U1 vocally utters information regarding the specific file. For example, the user U1 utters "daily report file" (C126). The interface device 10 converts the voice-recognized message into "nippou" corresponding to the "daily report" file, and sends the command "DOC nippou" to the processing device 2 (M122). The interface device 10 may also issue a voice saying "Okay!" to the user U1 (C128). Furthermore, the interface device 10 may change the display mode of the character (V126).

In this way, by having several back-and-forth conversations between the user and the character, the content of the process to be executed by the processing device 20 can be appropriately specified depending on the content of the conversation. Furthermore, when the content of the message uttered by the user is not clear, the interface device 10 may clarify the content of the message by repeating the conversation.

Note that what is displayed on the interface device 10 is preferably a character whose state can be expressed through facial expressions, movements, postures, and the like. For example, FIG. 10 shows an example of a character displayed on the interface device 10, in which a character imitating a human is displayed. FIG. 10(a) shows the character V130 as the state of the first character. Further, FIG. 10(b) shows the character V132 as the state of the second character.

For example, character V130 in FIG. 10(a) shows a normal state. Further, the character V132 in FIG. 10(b) shows an angry or troubled state. The states of these characters may be expressed mainly by a combination of movements (actions) such as facial expressions, movements, and postures of the characters. The character's facial expression can be expressed by the character's face. Furthermore, the character's motion and posture can be expressed by the character's body parts. Furthermore, the character's emotions can be expressed from the character's facial expressions, actions, and posture.

For example, when transitioning the state of a character from FIG. 10(a) to FIG. 10(b), it is necessary to transition from a normal expression as in the face part F102 to an angry expression as in the face part F104. good.

Note that various facial expressions are possible. FIG. 10(c) is a diagram schematically showing facial expressions. The facial expression can be switched to various expressions such as a happy expression F112, an angry expression F114, a neutral expression F116, a troubled expression F118, a crying expression F120, and a happy expression F122.

Also, when transitioning the facial expression of the character from FIG. 10(a) to FIG. 10(b), the character's facial expression is transitioned from a normal state like body part B102 to an angry action like body part B104. You can.

Note that various body movements and postures can be considered. For example, various motions such as a peace sign, a cheer, a shout, a running motion, and a sleeping motion may be shown. Further, the character's motion may be not only a still image but also a moving image. For example, an action in which the character is tired may be shown by a moving image in which the character is out of breath, or may be shown in a moving image in which the character is running. Moreover, the character may be a character imitating an animal or a device in addition to a character imitating a human being.

[1.6 Effect]
As described above, according to the present embodiment, by using the interface device 10, the processing device 20 can realize a system in which a user inputs instructions for executing a process using characters using few resources.

For example, focusing on a character, the character's "heart" (for example, controlling conversation with the user, expressing the character's emotions, etc.) can be implemented in the interface device 10. Additionally, the character's "ears" can also be implemented in the interface device 10. Furthermore, the character's "mouth" may also be implemented in the interface device 10.

As a result, it is possible to provide a system that is highly effective and user-friendly, without using the resources of the processing device 20 (for example, a PC) as much as possible.

For example, a user may issue an instruction to the interface device 10 such as "Search for the spreadsheet file you were looking at yesterday." The interface device 10 makes a response such as "OK". Further, the interface device can issue instructions (commands) to the processing device 20 in addition to responding. Note that the interface device 10 may issue a command at the same time as issuing a response. As a result, the "spreadsheet file that was viewed yesterday" is displayed on the processing device 20.

Furthermore, according to this embodiment, the interface device 10, which is an external device, is the controlling side, and the processing device 20 is the controlled side. A command that directly uses the API of the OS 2200 of the processing device 20 may be transmitted from the interface device 10 .

Further, the interface device 10 is preferably a dedicated device rather than a general-purpose device. For example, the interface device 10 uses the above-mentioned Air touch sensor so that the user can feel as if he or she is touching the character. Further, the interface device 10 may have an interesting external appearance.

Further, the interface device 10 may be a dedicated terminal of a size that is portable enough to be equipped with a display and a lightweight OS such as Android (registered trademark), Windows Embedded (registered trademark), or Chrome OS.

Additionally, the above-described functionality may be incorporated into other applications. For example, by incorporating it into a game application in which characters play an active role, it becomes possible to check the status of the character while performing game operations.

[2. Second embodiment]
A second embodiment will be described. The second embodiment is an embodiment in which the user is unable to make a sound. Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

In the second embodiment, the interface device 10 has a function of receiving operation input from a user by means other than voice, remotely controlling the processing device 20, and outputting an output associated with a character. The interface device 10, for example, uses the input unit 150 to receive inputs from the user such as button operations and touch operations on physically provided input means. Note that operations other than voice operations from the user are referred to as physical operations. The physical operation includes physically selecting a hardware button, selecting a software button displayed on the display unit 240, and the like. Further, the physical operations may include operations based on gestures, which are movements of the user.

Further, in the command table 2200 of the second embodiment, physical operations are associated with commands. For example, in the command table 2200, the command "MAIL" is associated with a corresponding operation button (for example, any operation button provided on the interface device 10). Further, the command table 2200 may associate gesture motions as physical operations with commands.

In the command table 2200, one command may be associated with a plurality of physical operations. Further, the command table 2200 may associate one command with the message content shown in FIG. 5(a) and further with a physical operation.

In the recognition process (step S102) in FIG. 6, in this embodiment, a command is generated by a physical operation other than voice from the user.

For example, the message interpretation unit 1010 shown in FIG. 3 accepts a physical operation input using the input unit 150 instead of a message that is the result of voice recognition performed by the voice recognition unit 1100. The command generation unit 1020 then generates a command based on the received physical operation.

The interface device 10 may display operation buttons on the display unit 140, for example, in order to accept physical operations from the user. When the user selects an operation button displayed on the display unit 140, it is determined that a physical operation has been performed on the input unit 150.

Further, the interface device 10 may further include a physical operation button as the input unit 150 in order to receive a physical operation from the user. The interface device 10 determines that a physical operation has been performed on the input unit 150 when a physical operation button is selected by the user.

Further, the interface device 10 may be a device that can receive input of a user's gesture as a physical operation using the input unit 150, and can recognize this gesture. For example, the input unit 150 can receive operations according to user gestures by recognizing a human skeleton and facial expressions using a depth sensor or a camera. As a device capable of recognizing user gestures, for example, Microsoft's Kinect or the like may be used.

In this way, in the second embodiment, the interface device 10 may be configured to accept not only voice input from the user but also input other than voice. The interface device 10 may generate commands based on physical operations that are inputs other than voice inputs. The user can execute processing in the processing device 20 by not only operating the processing device 20 by voice but also by performing non-voice operations on the interface device 10 .

Furthermore, the user can use voice operations and physical operations simultaneously or by switching between them. In addition, in the case of switching between the voice operation and the physical operation, the user may switch between the voice operation and the physical operation at any timing, or depending on the time of day. Further, it may be switched depending on the installation location of the interface device 10. For example, the interface device 10 may acquire the installation location of the interface device 10 using a location information acquisition means, and may switch between voice operation and physical operation depending on the location. Here, the position acquisition means may acquire the position using, for example, GPS (Global Positioning System), or may acquire the position depending on the wireless connection destination to which the interface device 10 connects. For example, when the wireless connection destination is a mobile communication system, base station information transmitted from the base station device may be used. Furthermore, when the wireless connection destination is a wireless LAN, the location may be specified using the SSID transmitted from the base station.

[3. Third embodiment]
[3.1 Configuration]
A third embodiment will be described. The third embodiment is an embodiment in which operation input can be performed on the processing device 20 in addition to operation input from the interface device 10. Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

FIG. 11 shows the software configuration of the third embodiment. The processing device 20 of the third embodiment further includes a chat control section 2400.

Chat control unit 2400 displays the operation input received by operation unit 250 from the user on display unit 240 as a chat screen. Here, the operation may be, for example, a method in which the user selects from among the selection candidates displayed on the display unit 240, or a method in which the user inputs a character or a character string using a hardware keyboard or a software keyboard.

Chat control section 2400 transmits the character string input by the user to conversation control section 1300. Conversation control unit 1300 transmits a character string in response to the character string input by the user to chat control unit 2400.

Chat control unit 2400 displays the character string input by the user and the character string acquired from conversation control unit 1300 on the chat screen.

Here, the chat screen is a display screen on which a conversation controlled by the conversation control unit 1300 is displayed. For example, the screen displays a character string input by the user and a character string output by the conversation control unit 1300 in chronological order.

Further, the chat screen does not need to be displayed on the screen all the time. For example, the display screen may normally be stored in the task bar or minimized. Then, when it is necessary to chat, a pop-up process may be performed as necessary, the window may be restored, and the chat screen may be displayed on the desktop.

Although the chat control unit 2400 displays the chat screen on the display unit 240, it may display the chat screen on the display unit 140 or both.

That is, the interface device 10 acquires the operation input by the user via the processing device 20. The interface device 10 also outputs a message to the user via the processing device 20.

Further, in the interface device 10, the conversation control unit 1300 may change the display mode of the character depending on the content of the conversation with the user. That is, when the interface device 10 transmits a character string to be used in a conversation to the processing device 20, it causes the character to make an appropriate reaction. Since the character's facial expression, posture, and motion change appropriately depending on the content of the conversation, the user can create the feeling that the character is performing the processing.

[3.2 Process flow]
FIG. 12 is a diagram showing the processing of this embodiment, replacing FIG. 6 of the first embodiment. In FIG. 12, processing for executing chat processing (step S202) is added.

When the interface device 10 or the processing device 20 (hereinafter referred to as the interface device 10, etc.) needs to chat, it starts chat processing (step S202). For example, the interface device 10 may send a command or a trigger signal to the processing device 20 for executing chat processing in the processing device 20. Furthermore, the processing device 20 executes the chat process when receiving a command to start the chat process or a trigger signal from the interface device 10 .

Further, the processing device 20 may perform chat processing when the user performs an operation. When the processing device 20 executes the chat process, it may transmit to the interface device 10 that the chat process has been executed.

When chat processing is started in the processing device 20, a character input mode may be started in the processing device 20. In the character input mode, it is preferable that the conversation in which the user inputs characters and the conversation spoken by the character are displayed in a conversation format. Here, the conversation spoken by the character may be characters output by the conversation control unit 1300 of the interface device 10. Furthermore, the interface device 10 may change the display mode of the characters on the interface device 10 when the characters are having a simulated conversation.

Further, the interface device 10 and the like may be configured to cancel chat processing. Chat processing may be stopped by performing an end operation on the processing device 20. Further, the chat processing may be stopped by detecting that the chat processing is to be ended in the interface device 10 and transmitting a command or a trigger signal to the processing device 20 to stop the chat processing.

When the chat process is canceled, the user can operate the processing device 20 by inputting voice to the interface device 10, as in the first embodiment.

Furthermore, the interface device 10 and the like may accept voice input from the interface device 10 even when chat processing is started.

Here, the interface device 10 and the like determine whether it is necessary to start chat processing depending on the environment of the interface device 10. Here, some examples of determining that it is necessary to start chat processing will be described.

(1) Determining based on the user's operation to perform a chat If it is detected that the user has performed an operation to perform a chat, the interface device 10 etc. start chat processing. For example, the user may perform an operation on the interface device 10 to start a chat. This operation may be performed by the user making a sound or by using an operation button.

Further, the user may perform an operation on the processing device 20 to start a chat. For example, the user may operate a button displayed on the processing device 20 to start a chat. Further, the user may start a chat by performing a key input from the processing device 20.

Further, the user may perform an operation to switch from chatting to voice input via the interface device 10. Thereby, the chat process ends, and the user can input voice operations to the interface device 10 as in the first embodiment.

(2) Determination based on the location of the interface device 10 etc. The location where the interface device 10 is installed may be detected by the location information acquisition means, and chat processing may be executed as necessary. The location information acquisition means for acquiring the location of the interface device 10 may use, for example, GPS (Global Positioning System) to acquire the location, or may acquire the location depending on the wireless connection destination to which the interface device 10 connects. good. For example, when the wireless connection destination is a mobile communication system, base station information transmitted from the base station device may be used. Furthermore, when the wireless connection destination is a wireless LAN, the location may be specified using the SSID transmitted from the base station.

If the interface device 10 is at the location to which the user connected in the previous chat process, the chat process may be started. For example, the interface device 10 corresponds to the location where the chat process was executed last time, for example, the latitude and longitude information when the chat process was executed, and the SSID (an example of base station information) when the chat process was executed. and associate chat processing. If the interface device 10 is located at a nearby location indicated by the associated latitude and longitude information, a chat process may be started. Furthermore, when the interface device 10 is connected to the base station to which it was connected when it started the chat process last time, it starts the chat process.

Further, the interface device 10 may start chat processing, for example, when the location where the interface device 10 is installed is a public facility such as a company, library, or restaurant. Further, the interface device 10 may start chat processing when the user is traveling by train, for example. Further, the interface device 10 may end the chat process, for example, when the user is traveling in a car (for example, a private car).

(3) Determination based on schedule A schedule for starting chat processing may be registered in the interface device 10 or the like. The interface device 10 and the like may refer to the registered schedule and start the chat process if the time is included in the time to start the chat process. For example, the user may register late at night as the time to switch to chat processing because the sound becomes louder.

(4) Determine according to the environment If the interface device 10 or the like determines that it is appropriate to start the chat process, it may execute the chat process. For example, when the surrounding environmental sounds are below a predetermined volume level, the interface device 10 may determine that the facility is a public facility and perform chat processing. Further, when the user is talking in a low voice, the interface device 10 or the like may determine that the user is in a place where he or she cannot speak, and may switch to chat processing.

(5) Determine according to the learning result The interface device 10 etc. performs learning according to the number of times the chat processing is switched, determines whether to start the chat processing according to the learning result, and executes the chat processing. You may.

Note that switching to these chat processes may be confirmed by the user, or may be automatically switched. Further, the interface device 10 may ask the user, "Is it OK to chat today?"

[3.3 Operation example]
Next, the operation of this embodiment will be explained with reference to FIG. 13. For example, it may be determined that the interface device 10 performs chat processing. The interface device 10 transmits a trigger signal to the processing device 20 to execute chat processing.

Further, an operation may be performed on the processing device 20 to execute a chat process. At this time, the processing device 20 may transmit a command or a signal indicating that the chat process has been executed to the interface device 10.

Then, the processing device 20 displays a chat window in the area R202 on the display screen W202 by executing the chat process. The chat window displayed in area R202 displays character strings (C202, C206) acquired from the operation unit 250 and character strings (C204, C208) acquired from the interface device 10. Further, in addition to the acquired character string, the chat window may also display an identification display indicating the user, the date and time when the character string was acquired, etc. Further, the identification display may be, for example, an icon indicating the user or a user name. Further, the identification display may be displayed with a different font type or character color for each user.

When the character string input by the user is transmitted to the interface device 10, the interface device 10 executes recognition processing on the character string. The recognition process for a character string is a process for identifying a command corresponding to the character string. If there is a command corresponding to the character string input by the user, the interface device 10 generates the command and sends it to the processing device 20.

The processing device 20 also interprets commands received from the interface device 10 and executes corresponding processing. For example, in this figure, when the user inputs the character string "Search for yesterday's files" (C202), the interface device 10 receives the character string. The interface device 10 interprets the character string "Search for yesterday's files" and generates a command to display a list of files dated yesterday. When the processing device 20 receives a command to display a list of files dated yesterday, the processing device 20 displays a list of files dated yesterday. As a result, for example, in area R204 of display screen W202, a window displaying a list of files dated yesterday is displayed.

Note that although the chat window is displayed on the processing device 20 in the above-described embodiment, it may be displayed on the interface device 10. Further, the chat window may be displayed on a device other than the processing device 20. The device other than the processing device 20 may be, for example, a smartphone, a tablet device, or a wearable terminal device worn by the user.

In this way, according to the present embodiment, the present system can be used even in situations where voice input is difficult. For example, if voice input is difficult, such as in a company, text is input into the processing device 20 and transmitted to the interface device 10. By returning content in response to the input text from the interface device 10 to the processing device 20, it is possible to create a state in which it appears as if the user is chatting with a character. That is, the speech of the character on the interface device 10 can also be confirmed on the processing device 20.

Further, the mode for inputting voice from the interface device 10 (voice mode) and the mode for inputting characters from the processing device 20 (chat mode) may be automatically switched.

Note that the output associated with the character in the interface device 10 and the processing executable by the processing device 20 may be the same in the voice mode and the chat mode, but at least one may be different. By making at least one of them different, it can be expected that the user will be more motivated to use both the voice mode and the chat mode. Further, the relationship between the recognized character string and the command may be the same in the voice mode and the chat mode, or may be different. If they are the same, it will be easier to set the relationship between the character string and the command, and if they are different, it will be easier to set the relationship suitable for the characteristics of each mode, voice mode and chat mode.

[4. Fourth embodiment]
Next, a fourth embodiment will be described. In the embodiments described above, the interface device 10 has been described using as an example a device that displays characters on the display unit 140. In this embodiment, the interface device 10 is replaced with another device. Further, in this embodiment, an embodiment in which there are a plurality of processing devices 20 will also be described.

Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

[4.1 Interface device]
FIG. 14(a) is an image diagram of a doll-shaped interface device 12 imitating a character or the like. That is, the casing of the interface device 12 may have an appearance related to the character. The casing of the interface device 12 may be a doll-shaped device, or may have a shape that directly represents a character, or may have a deformed shape. Furthermore, the casing of the interface device 12 may be made of a material such as polyvinyl chloride, polyresin, or soft vinyl for its outer shape. Further, as the doll-shaped device, a product such as a so-called "Nendoroid" may be used.

Further, the interface device 12 may be provided with a notification device instead of the display section 140. In FIG. 14(a), the LED 242 is provided in the eye area. The LED 242 can indicate the status to the user by lighting up, blinking, or the like. Further, the interface device 12 may have a built-in speaker or microphone. Further, the interface device 12 may move its body parts or change its facial expression. The interface device 12 may be a robot device modeled after a character or the like.

Furthermore, the interface device 10 may be represented by a character displayed in a virtual space by being incorporated into VR goggles or MR goggles, instead of a shape represented by a specific casing.

Furthermore, the interface device 10 may be used to install an application on an existing device. FIG. 14(b) shows a wristwatch-type wearable terminal device 14 worn by a user. The wearable terminal device 14 may be, for example, a wristwatch type device (watch type device).

Moreover, FIG. 14(c) shows the smartphone 16. By installing the application 1000 shown in FIG. 3 and storing the command table 1200 in these devices, it is possible to realize functions equivalent to those of the interface device 10. Further, when an existing device provides an interface for displaying characters, it is preferable to further install an application corresponding to the conversation control section 1300 and the character control section 1400. Further, the speech recognition section 1100 and the information acquisition section 1500 may use other applications or may be installed together.

[4.2 Processing device]
The interface device 10 may be connected to multiple processing devices. For example, FIG. 15 is a diagram schematically showing that the interface device 10 can be connected to a plurality of processing devices. In addition to the processing device 20, the interface device 10 can also be connected to a processing device 22 as another processing device.

By connecting the interface device 10 with different processing devices, it is possible to share data, for example. For example, the interface device 10 receives response information received from the processing device 20 and stores it in the storage 110. Then, when the interface device 10 connects to the processing device 22, it transmits information included in the response information to the processing device 22. This makes it possible, for example, to exchange data between different terminal devices.

By configuring the interface device 10 so that it can be connected to a plurality of processing devices 20, the interface device 10 functions as a hub and can transmit and receive data. Furthermore, the user can feel as if the character is acting as a hub and exchanging data with the plurality of processing devices 20.

Further, the processing device may be connected to a device other than a computer. For example, the interface device 10 in FIG. 15 can be connected to the in-vehicle device 24.

Here, the in-vehicle device 24 includes a device that can be mounted and used in a vehicle. For example, here, the in-vehicle device 24 may be a car navigation device. In addition, the in-vehicle device 24 may be a device installed in a vehicle, such as a drive recorder, a full driving support device (for example, a radar detector or a laser detector), or a car audio. Further, the in-vehicle device 24 only needs to be usable in the vehicle, and is a concept that includes, for example, a smartphone and a tablet used in the vehicle.

Hereinafter, a case in which the interface device 10, the processing device 20, and the vehicle-mounted device 24 cooperate to perform processing will be described.

For example, when a user asks the interface device 10, "Is there a delicious sushi restaurant near station A?", the interface device 10 converts it into a command and sends it to the processing device 20.

The processing device 20 interprets the received command and displays a map on the display unit 240. At this time, the processing device 20 may execute the search process based on the information input from the interface device 10. Additionally, the processing device 20 may display a list of store information together with the map.

The user speaks the desired store name into the interface device 10 from among the displayed names. The interface device 10 transmits a command including the spoken store name to the processing device 20. The processing device 20 acquires information regarding the store corresponding to the store name and displays it on the display unit 240. Furthermore, the processing device 20 includes store information including the location information of the store in the response information and transmits it to the interface device 10 .

The interface device 10 stores the store information included in the received response information in the storage 110. After connecting to the in-vehicle device 24, the interface device 10 transmits store information to the in-vehicle device 24. The car navigation device, which is the in-vehicle device 24, can be set to use the location information included in the received store information.

The interface device 10 also acquires information regarding driving from the in-vehicle device 24. Then, when the interface device 10 is connected to the processing device 20 in the home, the interface device 10 transmits a command based on information related to driving to the processing device 20. The processing device 20 may display, for example, the location where the dangerous driving occurred on the browser. Furthermore, the interface device 10 may notify the user, ``Dangerous driving occurred'' via the character.

Further, the interface device 10 may be connected to a plurality of processing devices 20. At this time, the interface device 10 may display a different character for each connected processing device. For example, the user may specify the processing device to be operated by touching the character. Alternatively, the user may specify the processing device to be operated by calling the character's name during conversation. Further, at this time, it may be possible to specify what the processing device 20 can do depending on the state of the character. For example, in the case of a character wearing a hat, it may be possible to specify the character based on its items or characteristics, such as controlling the power supply of the processing device.

[5. Fifth embodiment]
[5.1 Overview]
Next, a fifth embodiment will be described. The fifth embodiment is an embodiment in which display can be performed on the interface device 10 according to the state of the processing device 20. Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

Here, the device state is a state that can be acquired by the processing device 20. For example, it indicates the state of the resources of the processing device, and the states that can be obtained differ depending on the type of the processing device 20. In this embodiment, the interface device 10 outputs the state of the processing device and the character in association with each other. The following are examples of the device status that can be obtained.

(1) Information processing devices such as computers For example, the device status includes the status of computer resources. For example, the status of the device includes the remaining battery level, the load factor of the device, the remaining capacity of storage, the remaining capacity of memory, the amount of network transfer, and the like.

The states of these devices may be associated with the states of the characters. For example, when the remaining capacity of the storage is low, a character with a sad expression may be output, and when the remaining capacity of the storage is sufficient, a character with a happy expression may be output. The facial expressions and movements of the characters may be associated with each other depending on the state of the device.

(2) Mobile Battery The processing device 20 may be, for example, a device such as a mobile battery. When the processing device 20 is a mobile battery, the states that can be acquired as the device state include battery states such as remaining battery power, current power consumption, and remaining charging time. Further, the state of the battery may include a state in which the battery is being charged, a state in which the battery is being discharged, or the like. Further, the state of the battery may include the output voltage from the battery, the voltage at the time of charging the battery, and the like.

The processing device 20 may associate the state of these batteries with the state of the character. For example, a character with a distressed expression may be output when the battery level is low, or a character with a satisfied expression may be output when the battery level is sufficient. Further, when the battery is being charged, for example, a character pedaling a bicycle may be output.

(3) Vehicle-mounted device The processing device 20 may be, for example, a vehicle-mounted device such as a car navigation system, a safe driving support device (so-called radar detector or laser detector), or a drive recorder. When the processing device 20 is an in-vehicle device, the obtainable state is, for example, a state based on position information, information regarding the vehicle, information obtainable from an acceleration sensor such as a drive recorder, or the like. In addition, information regarding the vehicle includes, for example, battery voltage value, speed, number of revolutions, water temperature, information that can be obtained from other sensor devices, information that can be obtained via the vehicle's fault diagnosis connector, OBD (On-board diagnostics) This is information that can be obtained from.

The interface device 10 may associate the states acquired from these in-vehicle devices with the states of the characters. For example, when dangerous driving is detected, a character with a surprised expression may be output. Furthermore, the interface device 10 may change the facial expression of the character depending on the speed of the vehicle. Furthermore, the interface device 10 may make the character move to prompt a notification when it is time for an oil change or the like.

[5.2 Process flow]
Next, specific processing will be explained. FIG. 16 is a diagram in which the processing in FIG. 6 is replaced. In FIG. 16, device status display processing (step S300) is added to FIG. 6 described in the first embodiment.

The device status display process is a process for outputting the status of the processing device 20 in the interface device 10. Although the device status display process is executed first in FIG. 16, it may be executed when the user performs an operation. Here, the state of the processing device 20 is output based on the state of resources of the processing device 20, for example.

In the device status display process, the processing device 20 acquires the status of the device (step S302). The processing device 20 may obtain the state of the device, for example, when the user performs a voice operation on the interface device 10 to obtain device information. Further, the interface device 10 or the processing device 20 may be in an operation mode in which the state of the processing device 20 is appropriately acquired and transmitted from the processing device 20 to the interface device 10.

The processing device 20 transmits the acquired device status to the interface device 10 (step S304). The interface device 10 receives the device status from the processing device 20 (step S306). The interface device 10 executes output processing based on the acquired device state (step S308). In the output process, the display mode of the character displayed on the display unit 140 may be changed depending on the device state. Furthermore, in the output process, audio may be output from the audio output unit 162 depending on the device state.

In addition to the device status display process, the processes described in the first embodiment (steps S102 to S112) can also be executed.

Further, although the interface device 10 of this embodiment changes the display mode of the character based on the state of the processing device 20, it may be a device different from the processing device 20. For example, the interface device 10 may be connected to a mobile battery that is a portable battery separately from the processing device 20, and the display mode of the character may be changed depending on the state of the mobile battery. An example of operation will be described below.

[5.3 Operation example]
An example of operation will be described with reference to FIG. 17.

[5.3.1 Computer status]
A case where the processing device 20 connected to the interface device 10 is an information processing device such as a computer will be described with reference to FIG. 17. FIG. 17 shows the correspondence between the battery status of the processing device 20 and the character display. The interface device 10 acquires information regarding the processing device 20 via the information acquisition unit 1500.

For example, the interface device 10 acquires information on the remaining battery level of the processing device 20. Then, when the remaining battery power of the processing device 20 is low (for example, low), the interface device 10 displays the character V302 in a first state (for example, an angry expression, an angry action, etc.). At this time, when the interface device 10 acquires that the processing device 20 is being charged, it causes the second character V304 to transition to a state (for example, a state announcing that charging is in progress).

Further, when the interface device 10 acquires information that charging of the processing device 20 has been completed, it displays the character V306 in a third state (for example, a smiling expression, a state in which a fun action is performed, etc.).

In this manner, according to the present embodiment, the user can easily obtain the current state of the processing device 20 based on the facial expressions and actions of the character. Furthermore, according to the present embodiment, the user can enjoy acquiring information about the processing device 20 by looking at the state of the character, as opposed to simply displaying the remaining battery level.

Further, the processing device 20 may output audio in addition to displaying the above-mentioned characters. For example, when displaying the character V302, the processing device 20 may output a voice saying "The battery is low." Further, the processing device 20 may output a voice saying "I'm working hard on charging!" when displaying the character V304. Further, the processing device 20 may output a voice saying "Charging completed!" when displaying the character V306.

Furthermore, the interface device 10 may dynamically change the state of the character depending on the state of the battery. For example, the interface device 10 may display a tired expression as a character when the remaining battery power of the processing device 20 decreases. Furthermore, the interface device 10 may change the state of the character according to, for example, the CPU usage rate, memory usage rate, disk access status, or network communication status instead of the remaining battery level. Further, the interface device 10 may change the state of the hand character depending on the number of windows opened in the processing device 20 or the number of programs being executed.

Furthermore, as a method of changing the character's state, the interface device 10 may display a state in which the character is holding a heavy object, for example. The interface device 10 may display a character with an expression or state of holding a heavy object as the load on the processing device 20 increases.

Further, when the interface device 10 detects a sign of a failure from the information acquisition unit 1500, the character may notify the user. For example, if there is a possibility of a failure in the storage of the processing device 20, such as an SSD or HDD, the interface device 10 may notify the user via a character of signs of failure such as "It's about to get dangerous" or malfunctions of the operation. You can tell me. For example, the interface device 10 may determine storage failure based on obtainable information such as operating time and the number of bad sectors. Further, the interface device 10 may determine the possibility of a storage failure based on, for example, a failure detection function from the SSD.

Further, after notifying the user of the possibility of a failure, the interface device 10 may transmit a command corresponding to the failure to the processing device 20 in response to an operation from the user. For example, when a user asks, "What should we do?", the interface device 10 may send a command to open a storage manufacturer's website, or send a command to send a failure report email. It's okay.

In this way, the interface device 10 can monitor the state of the processing device 20 by acquiring information from the information acquisition unit 1500. Furthermore, the interface device 10 can notify the user of the status of the processing device via a character based on the status of the processing device.

[5.3.2 Mobile battery]
An example of operation when the device from which the interface device 10 acquires information is a battery will be described. The following operation example will be explained using a mobile battery, which is a small, portable battery, as an example. For example, when a mobile battery is connected to a bicycle, it can assist the bicycle by supplying power to the bicycle's motor. Moreover, the mobile battery can charge a smartphone by connecting it to the smartphone.

Note that the battery may be a different device from the processing device 20, or the processing device 20 may be replaced with a battery. Hereinafter, for convenience of explanation, the processing device 20 will be explained as a mobile battery.

When using a mobile battery, the interface device 10 may add effects to the character depending on the battery state. The interface device 10 anthropomorphizes the mobile battery, which is the processing device 20, and displays it as a character. It is preferable that the interface device 10 has a function of notifying the user, via a character displayed on the interface device 10, that the mobile battery is working hard to supply power.

For example, the interface device 10 may notify the charging/power supply status of the mobile battery in a form that is compared to calorie intake and consumption for the character. Further, the interface device 10 may output the character differently depending on whether the mobile battery is in a charging state or in a power supplying state. For example, the interface device 10 may output different words, facial expressions, and emotions depending on the charging power during charging. Here, the interface device 10 may change the output based on the charging current if the charging power is a constant voltage, such as in a standard micro USB. Further, if the charging power can be changed in voltage like USB Type C, the interface device 10 may determine the voltage and change the output according to the voltage.

Furthermore, the interface device 10 may output different words, facial expressions, and emotions depending on the power supplied to the mobile battery. For example, when the power supplied by the mobile battery is large, the interface device 10 may output "It's hard to run at full speed." For example, the interface device 10 may output the state of the mobile battery as a character's feelings in words, voice, or facial expressions as the output content. Furthermore, the interface device 10 may output an explanation of the feelings of the mobile battery from a third person's perspective.

Further, if the mobile battery is connected to the processing device 20, the interface device 10 may acquire the state of the mobile battery via the processing device 20.

[5.3.3 In-vehicle equipment]
An example of the operation when the interface device 10 connects to an in-vehicle device will be described. The in-vehicle device may be, for example, a drive recorder, a car navigation device, a safe driving support device (so-called radar detector), or the like. Further, the in-vehicle device may replace the processing device 20, or the processing device 20 may be a different device. Hereinafter, the in-vehicle device will be explained as a processing device 20.

The interface device 10 acquires the status of the in-vehicle device from the in-vehicle device via the information acquisition unit 1500. Here, the interface device 10 may acquire facility information or position information from a car navigation device, for example, as the state of the in-vehicle device. Further, the interface device 10 may acquire the current driving state from a drive recorder.

For example, when a user says "I'm hungry" to the interface device 10, the interface device 10 sends a command to the car navigation device to search for a restaurant. The interface device 10 then obtains information on nearby restaurants as response information. The interface device 10 obtains information on nearby restaurants through the character. Furthermore, the processing device 20 (in-vehicle device) executes a search process in response to a message from a user. Then, the result of executing the search process may be returned to the interface device 10 as response information.

This allows the user to obtain information using the same interface regardless of the user interface of the car navigation device. For example, when a user uses a rental car, the car navigation device installed in the car used is provided with a different user interface depending on the manufacturer and model. Even if a different user interface is provided as a car navigation device, the user can obtain desired information simply by talking to the character via the interface device 10 that the user always uses.

Furthermore, when acquiring information from the vehicle-mounted device, the interface device 10 may acquire vehicle information via the vehicle-mounted device. For example, when the in-vehicle device can acquire vehicle information via OBD, the interface device 10 can acquire vehicle information.

Thereby, for example, the interface device 10 may notify the user of information regarding vehicle maintenance via the character. Further, the interface device 10 may notify the user of information regarding safe driving via a character. For example, when the character continues to speed excessively, the interface device 10 may output a message such as "I want to drive a little slower" from the character, and make the character look sad.

Further, the interface device 10 may make the character smile when safe driving continues or when there are no problems with the vehicle condition. Further, the interface device 10 may make the character look sad when dangerous driving is detected or when there is an abnormality in the vehicle condition.

Thereby, for example, the user can grasp the current driving state and the state of the vehicle just by looking at the character while driving. Furthermore, the interface device 10 may send the notification from the character when the vehicle is stopped.

[5.3.4 Computers and auxiliary equipment]
Further, the interface device 10 may connect the processing device to a device different from the processing device. The device different from the processing device is a device having a function different from that of the processing device, and may be, for example, an auxiliary device having a function to assist the processing device. The functions different from those of the processing device are functions that the processing device does not have or functions that assist the functions of the processing device. For example, if the processing device is not equipped with a communication module for mobile communication such as 4G/5G, a device with a function different from that of the processing device (e.g., mobile wireless router). Further, when a device having a function different from that of the processing device supports the battery of the processing device, it is preferable to use a mobile battery that supplies power to the battery of the processing device.

The auxiliary device different from the processing device may be a device having a single function. A device with a single function is a device that has a certain specific function and is used exclusively as a device that performs that one function. The battery 28 is a device that has a battery function (that is, a battery function) as one specific function, and is an example of a device that exhibits the battery function.

Hereinafter, a computer 26 as a first processing device is used instead of the processing device 20, and an auxiliary device different from the processing device 20 described above is used as a battery as a second processing device. A case where it is set to 28 will be explained. For example, in FIG. 18, the interface device 10 is connected to the computer 26 as a first processing device. Further, the interface device 10 is connected to a battery 28, which is an auxiliary device, as a second processing device. The battery 28, which is an auxiliary device, is a device that can supply power to other devices, and in FIG. 18, it supplies power to the computer 26, which is the first processing device.

The interface device 10 in FIG. 18 displays the state of the computer 26 in correspondence with a human character V402. Further, the interface device 10 displays the state of the battery 28 as a character V404 imitating a dog. Character V404 is Teru, a character used by Jupiter Corporation.

The interface device 10 displays the status of each processing device as different characters. Therefore, the user can check the status of each device by looking at the interface device 10. For example, the facial expression of the character V404 changes depending on the remaining amount of the battery 28. For example, when the battery 28 has sufficient remaining power, the character V404 has a "happy" expression. Furthermore, when the remaining battery power is low, the character V404 has a "sad" expression.

This allows the user to check not only the status of the computer 26 but also the status of the battery 28. Note that the interface device 10 may obtain the state of the battery 28 via the computer 26.

Note that the auxiliary device may be, for example, a microphone as an auxiliary device having a function of being connected to the processing device. For example, the interface device 10 may change the state of the character corresponding to the auxiliary device depending on the audio level acquired by the microphone.

Further, the auxiliary device may be, for example, a mobile wireless router as an auxiliary device having a function of being connected to the processing device. For example, the interface device 10 may change the state of the character corresponding to the auxiliary device depending on the communication status of the mobile wireless router.

In this manner, the user may, for example, recognize the desired states of the plurality of devices simply by visually recognizing the character, without specifically gazing at the interface device. Furthermore, the different characters may be configured as characters that are related to the processing device 20 and a device having a single function, for example. For example, the character corresponding to the processing device 20 may be displayed as an anthropomorphic character, and the character of the device having a single function may be displayed as a pet of the anthropomorphic character. With such a configuration, the user can recognize the relationship between the processing device and the device having a single function. Specifically, if the processing device is displayed as a "person" and the mobile battery is displayed as a "dog," the user's main character is the anthropomorphic character "human," and the user is treated as a pet of the anthropomorphic character. "Dog" may be displayed.

As described above, the interface device 10 preferably outputs the state of the processing device 20 in an anthropomorphic form using a character, but in addition to this, it is also possible to associate an image indicating the state of the processing device with an image of a character. It may also be used as output. Such output includes an output in which the interface device 10 displays a character holding an object (for example, a board-shaped object) that displays the state of the processing device 20. The output associated with the character may be an output in which the voice of the character is associated with the voice indicating the state of the processing device 20. Such output includes an output that guides the status of the processing device 20 using a character voice.

[6. Sixth embodiment]
A sixth embodiment will be described. The sixth embodiment is an embodiment in which a user generates a setting table 2300 and a command table 1200. Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

FIG. 19 is a flowchart for explaining command setting processing in this embodiment. This process is realized by the control unit 200 of the processing device 20 reading and executing a program stored in the storage 210.

When the user instructs the processing device 20 to execute the command setting process (step S402; Yes), the control unit 200 executes each step of the command setting process.

First, the control unit 200 receives input of operation details from the user (step S404). As the operation content, keywords included in the operation (for example, voice operation, etc.) may be registered. Further, as the operation content, words and character strings operated by the user may be registered. The user can execute the corresponding process by speaking to the interface device 10 some or all of the contents registered in the operation contents.

Subsequently, the control unit 200 registers the processing details (step S406). When registering processing details, for example, a program to be executed and parameter (argument) settings for the program to be executed may be registered by selection or character input. Further, in the processing device 20, the user may actually perform an operation, and the content of the operation may be registered as the processing content.

Next, since the operation content and the processing content are ready, the control unit 200 generates a command corresponding to these two contents. The command may be created arbitrarily by the control unit 200 or by the user. Then, the user can register one or more of the above-mentioned contents as necessary (step S410; No→step S404).

When all settings are completed (step S410; Yes), the control unit 200 updates the setting table 2300 (step S412). The control unit 200 also updates the command table 1200 stored in the interface device 10 (step S414).

Specifically, the control unit 200 transmits the operation details and commands among the above-mentioned items to the interface device 10. The interface device 10 updates the command table 1200 based on the received operation details and commands.

Furthermore, if the content of what the user has said to the character of the interface device 10 cannot be interpreted, the interface device 10 may perform an operation to register the operation. For example, when the user says, "Do XX," but the interface device 10 cannot interpret it, the interface device 10 prompts the user to register the process, saying, "I don't understand, so try it." When the user executes a process to be registered in the processing device 20, the control unit 200 can register the process content corresponding to the phrase "○○" based on the process. The control unit 200 may acquire an operation signal indicating an operation on the operation unit 250 performed after prompting registration of the process, and may register the process content based on the operation signal. For example, the control unit 200 specifies the processing content based on an operation signal indicating an operation performed within a predetermined time after prompting registration or after starting acquisition of an operation signal indicating an operation on the operation unit 250. , you may register it. Alternatively, the control unit 200 may register the process content based on a command generated in response to an operation on the operation unit 250 performed after prompting the user to register the process. This allows the user to feel that the characters displayed on the interface device 10 can now interpret things that could not be interpreted before.

[7. Seventh embodiment]
A seventh embodiment will be described. The seventh embodiment is an embodiment in which processing changes based on the content of the user's use of the system and the history of use. Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

In this embodiment, the interface device 10 and/or the processing device 20 may store history information. The history information is stored, for example, in the storage 110 of the interface device 10 or the storage 210 of the processing device 20.

The history information may include, for example, the number of times the user has spoken at the interface device 10. Further, the history information may include, for example, the number of times the processing device 20 has executed a process via the interface device 10. Further, the history information preferably includes the date and time when the processing device 20 executed the process, and the content of the process. Further, it is preferable that the history information includes contents inputted into the interface device 10 by the user. Further, the history information may preferably include a predetermined keyword among the contents input by the user.

In this embodiment, the processing device 20 can implement the following processing by using history information.

(1) Releasing restricted functions For example, the number of functions that can be executed by the processing device 20 may increase depending on the number of times of history, or the user may set the functions arbitrarily. For example, the processing device 20 can only execute a specific process up to three times in history. The processing device 20 can only execute processes stored in advance in the setting table 2300 in the processing device 20 until the number of times in the history is 10. The processing device 20 may be configured such that the user can register a new process in the setting table 2300 if the history count is 11 or more. That is, the types of input that can be made by the user may change depending on the user's history.

With such a configuration, the user can be motivated to use the interface device 10 more often. For example, when the interface device 10 is used as an advertising medium, there is an effect that the interface device 10 can be used preferentially.

Furthermore, since the things that can be done increase depending on the number of times the user uses the device, the user will be able to use new functions from time to time. That is, it becomes possible for the user to learn the functions in stages according to the user's level of proficiency with the interface device 10.

Furthermore, if the user is a minor, the processing device 20 may initially restrict the functions that can be used. For example, the processing device 20 may set a restriction such that the browser is not activated until the number of times the user uses it is 10 times. If the user is engaged in an appropriate conversation, the interface device 10 may accept a phrase that will launch the browser if the number of uses exceeds 10.

(2) Changing the character For example, the interface device 10 may increase the number of characters that can be used or change the appearance of the character depending on the number of times the user has made a history. For example, the interface device 10 may display one character as a character until the history number is 100, but the number of usable characters may increase by one each time the history number exceeds 100. Further, when the type of character is paid, the interface device 10 may process a discount if the number of history is large.

Further, the interface device 10 may increase the number of conversations between the characters or the number of characters displayed depending on the number of times the user has logged. For example, the interface device 10 may cause the character to respond in a different way, such as ``Okay!'' when the number of history history is small, but ``Okay!'' as the number of history history increases. You may also do so. Furthermore, the character's costume may change or accessories may be added depending on the number of times of history. Furthermore, the character may evolve. When a character evolves, for example, in the case of an anthropomorphic character, the character may change from a child to an adult. In this way, the output of the character may be switched depending on the user's history of use.

In this way, according to the present embodiment, the number of types of input by the user increases, the number of functions that can be executed increases, and the output characters may change as the number of times the user uses the app increases.

Further, the usage history indicated by the history information may include usage intervals, usage time, etc., in addition to the number of usages. Further, the usage history may be a usage history of the entire interface device, a usage history of each character, or a usage history of an application. For example, since the output of the character changes depending on the usage history, this can motivate the user to use the system more often.

[8. Eighth embodiment]
An eighth embodiment will be described. The eighth embodiment is an embodiment in which specific information is stored as information necessary for the interface device 10. Note that the system configuration and the configuration of each device of this embodiment are almost the same as those of the first embodiment, and the following description will focus on the differences in this embodiment.

For example, the interface device 10 stores specific information required when the processing device 20 executes the execution application 2600 in the storage 110.

Here, the specific information may be, for example, personal information. The personal information may include, for example, information such as address, name, date of birth, and settlement information such as card information. Further, the necessary information may be setting information in the execution application 2600. For example, the setting information may be information used to customize the execution application 2600. Further, the setting information may be license information of the execution application 2600. Further, the required information may be authentication information. The authentication information may be, for example, a login ID for a network service or a password. Further, the authentication information may be token information.

The processing device 20 may inquire of the interface device 10 for necessary information when executing the command execution process, for example in step S154 in FIG. 6. For example, the processing device 20 sends a signal to the interface device 10 requesting necessary information. The interface device 10 reads the requested information from the storage 110 and sends it to the processing device 20.

Since the necessary information is stored in the interface device 10, there is no need to store information in the processing device 20. Therefore, there is no need to store personal information or the like in the processing device 20, and a system that takes security and privacy into consideration can be provided. Further, since necessary information is stored in the interface device 10, for example, when there are a plurality of processing devices 20, the execution application 2600 can be executed in each processing device 20 based on the setting information. Even if the processing device 20 is different, the user can use the same settings via the interface device 10.

[9. Application example]
Application examples of the embodiments described above will be described below.

(1) Use in a computer The processing device 20 may be, for example, a computer at home. By connecting the interface device 10 to a computer at home, processing can be executed on the computer via the interface device 10. For example, a user can enjoy using a computer by displaying his/her favorite character.

(2) Use in remote work The processing device 20 may be a company-provided computer used in remote work. At this time, the interface device 10 may display, for example, a company mascot character as the character. Since the application 2000 has only the minimum necessary functions implemented, it does not limit memory or use many CPU processing resources even when executed on a computer.

Additionally, the interface device 10 can acquire the status of the processing device 20. The interface device 10 may transmit the status of the processing device 20 to, for example, a management server. By acquiring the status of the processing device 20, the management server can grasp, for example, the working status of employees in remote work.

Further, the interface device 10 may perform an action in which a character calls for attention depending on the state of the processing device 20. For example, when a user is viewing a homepage unrelated to work in a browser on the processing device 20, which is a company device, the interface device 10 may display a character saying, ``It doesn't matter,'' or ``You're not looking too much.'' You may also create a performance that calls for attention.

Furthermore, in this system, when the interface device 10 and the processing device 20 are connected through communication such as BLE, it is possible to operate the processing device 20 via the interface device 10 without using an external network. It becomes possible. Therefore, the interface device 10 can be connected to the processing device 20 with security ensured.

Furthermore, in a remote work system, it may be possible to change the character of the interface device 10. For example, the interface device 10 may display a company character, a mascot character, or the like.

(3) Use in education The processing device 20 may be a computer used in a school or educational institution. At this time, the person who uses the interface device 10 and the processing device 20 is a child who is a student. Therefore, by using the interface device 10, a child user can execute the processing of the processing device 20 via the character. The user operates the processing device 20 by talking to the character, and thus operates the processing device 20 while feeling familiar with the character.

(4) Recommendation Processing As described in the above embodiment, recommendations may be made by exchanging conversations multiple times between the user and the character (interface device 10).

For example, the user speaks to the interface device 10, "Start the browser." The interface device 10 asks, "Is there anything you want to see?" Then, the user speaks to the interface device 10, ``What is the hot topic now?''.

The interface device 10 transmits a command to the processing device 20 to “output what is being talked about.” The processing device 20 can collect trending information by visiting news sites and SNS based on commands. Furthermore, the processing device 20 may collect information from various sources or multiple sources for the same news. For example, information may also be collected from SNS (for example, Twitter, Instagram, FaceBook, etc.) as sites that could not be collected by the user alone.

Furthermore, the processing device 20 may transmit the collected results to the interface device 10 as response information. The interface device 10 preferably changes the output of information associated with the character in accordance with this response information, and particularly preferably changes the facial expression of the character. For example, in the case of sad news, the interface device 10 may change the facial expression to a sad face and read out the news in a calm voice. Further, in the case of bright news, the interface device 10 may change the facial expression to a happy face and read out the news in a bright voice.

Further, while the interface device 10 is being used, the character's reactions such as conversation with the character, character's expression, posture, movement, etc. may be changed. For example, in the case of news that is viewed by many users, the interface device 10 may determine that the user is interested in the news and may make the character's reaction larger than usual.

(5) Equipped with an artificial intelligence inference unit The interface device 10 may further include an inference unit that utilizes artificial intelligence. The conversation control unit 1300 may have a conversation with the user using the inference unit. Further, the character control section 1400 may change the state of the character using an inference section.

(6) Using a billing service This system may be applied to a billing service. For example, the interface device 10 and/or the processing device 20 connect to a billing server. Then, the interface device 10 and/or the processing device 20 may increase the number of available functions or remove restrictions depending on the user's current billing status.

For example, the interface device 10 may be configured such that only one processing device 20 can be registered for free, but it can be connected to a plurality of processing devices by paying a fee. Further, the interface device 10 may be released by charging users with the functions that can be used. For example, although the interface device 10 normally allows only voice input, it may be possible to use the chat function of the above-described embodiment by charging a fee.

Furthermore, the interface device 10 may allow the character to output advertisements at regular intervals instead of charging the user. Furthermore, when the character outputs an advertisement at regular intervals, the user may pay a fee to hide the advertisement.

Additionally, the interface device 10 may display various characters. For example, Vtubers, mascot characters, and anime characters may be displayed. For example, by distributing company characters for free, the system may provide advertising functionality via the interface device 10.

[10. Modified example]
The present invention is not limited to the embodiments described above, and various modifications are possible.
That is, embodiments obtained by combining appropriate modified technical means without departing from the gist of the present invention are also included within the technical scope of the present invention.

In addition, although the embodiments described above are described separately for convenience of explanation, they can be combined and executed to the extent possible.

Further, in each embodiment, the program that runs on each device is a program that controls a CPU, etc. (a program that causes a computer to function) so as to realize the functions of the above-described embodiments. The information handled by these devices is temporarily stored in a temporary storage device (for example, RAM) during processing, and then stored in various ROM or HDD storage devices, and read out and modified by the CPU as necessary.・Writing is performed.

Here, the recording medium for storing the program includes a semiconductor medium (for example, ROM, nonvolatile memory card, etc.), an optical recording medium/magneto-optical recording medium (for example, DVD (Digital Versatile Disc), CD (Compact Disc), etc. ), BD, etc.), magnetic recording media (for example, magnetic tape, flexible disk, etc.).

Furthermore, when distributing the program on the market, the program can be stored in a portable recording medium and distributed, or it can be transferred to a server computer connected via a network such as the Internet. In this case, it goes without saying that the storage device of the server device is also included in the present invention.

Furthermore, the data described above is not stored within the device, but may be stored in an external device and called up as appropriate. For example, data may be stored in NAS (Network Attached Storage) or on the cloud.

Note that the scope of the present invention is not limited to the configuration explicitly described in the specification, but also includes combinations of various aspects of the invention disclosed in this specification. Of the present invention, the structure for which a patent is sought has been specified in the attached claims, but at present, even if the structure is not specified in the claims, it is not disclosed in this specification. We intend to include such configurations in the scope of claims in the future.

The present invention is not limited to the configuration described in the embodiments described above. The components of each of the embodiments and modifications described above may be arbitrarily selected and combined. Also, any component of each embodiment or modification, any component described in the means for solving the invention, or a component that embodies any component described in the means for solving the invention. It may be configured in any combination. The applicant intends to acquire rights to these matters through amendments to the application or divisional applications. Even if there is a description of ``in the case of'' or ``in the case of'', the description is not intended to be limited to that case or at that time. We have also disclosed these cases and other configurations, and we intend to acquire the rights. Furthermore, the sections described in order are not limited to this order. It also discloses a configuration in which some parts have been deleted or the order has been changed, and we have the intention to acquire the rights.

In addition, the applicant intends to acquire rights to the entire design or partial design by converting the application to a design registration application. Although the drawing depicts the entire device using solid lines, the drawing includes not only the overall design but also the partial design claimed for some parts of the device. For example, it is a drawing that not only includes some members of the device as a partial design, but also includes some parts of the device as a partial design regardless of the members. The part of the device may be a part of the device or a part of the device. We intend to obtain rights not only for the entire design, but also for partial designs in which any part of the solid line part of the drawing is a broken line part. In addition, the modules, members, parts, etc. inside the device housing shown in the drawings are all subject to independent transactions, and similarly, they are included in the design registration application. There is an intention to make changes and obtain rights.

1 System 10 Interface device 100 Control unit; 110 Storage; 120 ROM; 130 RAM
140 Display section; 150 Input section; 160 Voice input section; 162 Voice output section 170 Communication section 1000 Application 1010 Message interpretation section; 1020 Command generation section 1030 Command transmission section; 1040 Response information reception section 1100 Voice recognition section 1200 Command table 1300 Conversation Control unit 1400 Character control unit 1500 Information acquisition unit 20, 22, 24, 26, 28 Processing device 200 Control unit; 210 Storage; 220 ROM; 230 RAM
240 Display section; 250 Operation section; 270 Communication section 2000 Application 2010 Command reception section; 2020 Command interpretation section 2030 Command execution section; 2040 Response information processing section 2200 Operating system (OS)
2300 Setting table

Claims (51)

comprising a processing device and an interface device that are communicably connected to each other,
The processing device includes:
a control unit that performs control to execute processing according to input from a user to the interface device;
The interface device includes:
a reception unit that receives the input;
an output unit that outputs information according to the input or the processing to the user by associating it with a predetermined character;
A system with The system according to claim 1, wherein the output unit outputs information according to the input or the processing by displaying an image of the character that changes according to the input or the processing. The system according to claim 2, wherein the image of the character that changes depending on the information according to the input or the processing is an image in which the facial expression of the character or the action of the character changes. The system according to claim 2 or 3, wherein the output unit outputs information according to the input or the processing using the voice of the character that changes according to the input or the processing. The processing device includes a display unit that outputs information by display,
The control unit outputs the results of the executed processing to a display unit on the processing device side,
The system according to any one of claims 1 to 4, wherein the output unit of the interface device outputs the fact that the process has been executed in association with the character. Any one of claims 1 to 5, wherein the processing device and the interface device are connected via a wireless communication path whose communicable distance is a predetermined distance or less, or connected via a wired communication path. system described in. The system according to any one of claims 1 to 6, wherein the output unit outputs the state of the processing device in association with the character. The system according to claim 7, wherein the output unit outputs the state of the processing device according to the state of resources of the processing device. The system according to claim 7 or 8, wherein the output unit outputs a battery state as the resource state of the processing device. The system according to any one of claims 7 to 9, wherein the output unit outputs the state of the processing device based on a change in the facial expression of the character or an action of the character. The reception unit receives a message from the user,
The system according to any one of claims 1 to 10, wherein the control unit executes processing based on the message or a phrase included in the message. The reception unit prompts the user to input a message again based on the message,
The system according to claim 11, wherein the control unit executes processing based on the message and/or the message received again. The interface device further includes an audio input section,
The system according to any one of claims 1 to 12, wherein the reception unit receives voice input from a user through the voice input unit. The interface device recognizes a message from the voice that has received the input,
The system according to claim 13, wherein the control unit executes processing based on the message or a phrase included in the message. The system according to claim 13 or 14, wherein the output unit changes and outputs the character according to the voice input by the user. The reception unit transmits a command corresponding to the received input to the processing device,
The system according to any one of claims 1 to 15, wherein the control unit receives the command and performs control to execute processing corresponding to the command. 17. The system of claim 16, wherein the command corresponds to an interface that utilizes functionality of an operating system of the processing device or an application executed on the processing device. 17. The control unit executes an interface that utilizes a function of an operating system executed in the processing device or an application installed in the processing device in response to an input from a user to the interface device. The system described in any one of the above. 19. A system according to any preceding claim, wherein the processing device obtains input from a user to the interface device by means of a resident application. 20. The system of claim 19, wherein the resident application receives commands from the interface device and obtains input from a user by interpreting the received commands. The system according to any one of claims 1 to 20, wherein the output unit outputs the state of the different device depending on the state of the device different from the processing device. The system according to claim 21, wherein the different device is a device having a different function from the processing device. The device having a different function from the processing device is a battery,
The system according to claim 22, wherein the output unit changes and outputs the character in association with the state of the battery. The system according to any one of claims 21 to 23, wherein the output unit outputs different characters between the processing device and a device having a function different from that of the processing device. The system according to claim 24, wherein the output unit displays a different character image or outputs a different sound for each character. The system according to any one of claims 21 to 25, wherein the output unit outputs the state in association with the state of a predetermined character. The interface device is further connectable with a different processing device,
The interface device includes:
The system according to any one of claims 1 to 26, wherein information corresponding to the processing acquired from the first processing device is transmitted for use when a different second processing device executes the processing. The system according to claim 27, wherein the information according to the process includes information regarding a search result when the first processing device executes the search process. The system according to claim 28, wherein the information regarding the search results is location information. the interface device stores specific information;
The system according to any one of claims 1 to 29, wherein the control unit acquires the specific information from the interface device and executes processing using the specific information. The system according to claim 30, wherein the specific information is any one of personal information, setting information when executing the process, and information regarding login. The system according to any one of claims 1 to 31, wherein the processing device further includes a setting unit that associates and sets the input from the user and the process to be executed. The system according to claim 32, wherein the setting unit associates a message or a specific phrase input by a user with the process to be executed. The system according to claim 33, wherein the specific phrase is a phrase that has no relation to the process to be executed. 35. The system according to claim 31, wherein the executed process is an application executed on the processing device or an interface that calls a function of an operating system. the interface device stores the command and the message;
When the reception unit receives a message input from the user, the reception unit transmits a command corresponding to the message to the processing device,
The system according to any one of claims 31 to 35, wherein the control unit executes processing corresponding to the command. The system according to any one of claims 31 to 36, wherein the setting unit specifies the process to be executed based on an operation input by a user in the processing device. 37. The setting unit sets a process specified based on the user's operation in the processing device in correspondence with the input when the process to be executed cannot be specified based on the user's input. The system described in any one of the above. The system according to any one of claims 1 to 38, wherein the output unit switches the output of the character according to a history of use of the interface device by the user. The system according to any one of claims 1 to 39, wherein the reception unit changes the type of input that can be accepted from the user depending on a history of the user's use of the interface device. The system according to any one of claims 1 to 40, wherein the control unit changes the processing to be executed depending on the history of the user's use of the interface device. The processing device has a character input mode that accepts character input from a user,
The output unit outputs information based on the input in the character input mode,
The system according to any one of claims 1 to 41, wherein the control unit executes a process instructed in response to input of the character input mode. 43. The system according to claim 42, wherein the character input mode has a function of realizing a conversational interaction between a user's character input and a character output by an interface device. The system according to claim 42 or 43, wherein the character input mode displays characters input by the user and characters acquired from the interface device in a conversational format. 45. The system according to any one of claims 42 to 44, wherein the interface device transmits a signal to the processing device to execute a character input mode depending on the environment of the interface device. 46. The system of claim 45, wherein the environment of the interface device is an environment based on the location where the interface device is installed. 47. The system according to claim 46, wherein the location where the interface device is installed is determined according to a connection destination to which the interface device is connected. 48. The system according to any one of claims 45 to 47, wherein the environment of the interface device is determined depending on the volume level of sound acquired by the interface device. The interface device includes a housing having the output section,
49. A system according to any preceding claim, wherein the housing has an appearance associated with the character. An interface device that is communicably connected to the processing device,
a reception unit that receives the input;
an output unit that outputs information based on processing according to the input to the user or the input from the user to the interface device by associating it with a predetermined character;
An interface device having: A program for causing a computer to function as a control unit of the system according to any one of claims 1 to 49.

Images