WO2020049826A1 - Information processing device - Google Patents

Abstract

An information processing device that comprises: an acquisition part that acquires content information about content; and an analysis part that, on the basis of the content information, analyzes user input that is made in natural language to an application that processes the content.

Description

Information processing device

<< The present invention relates to an information processing device.

2. Description of the Related Art There is known an information processing apparatus including a voice agent function of interpreting a voice input such as a voice command issued by a user and executing a process instructed by voice. For example, a voice input processing device that enables the use of simplified voice commands has been proposed (for example, Patent Document 1). This type of voice input processing device, for example, when a simplified voice command is received, with reference to an operation history that is a history of operation information that associates at least a part of the content of the voice command with the operation content, Issues predetermined commands for various controls.

JP-A-2017-146437

However, when an ambiguous instruction is issued from the user, the user's instruction may not be specified even by referring to the operation history. For this reason, in a conventional information processing apparatus that employs a voice agent function, when an ambiguous instruction is received from a user, a process intended by the user may not be performed in some cases. Therefore, the usability of the information processing apparatus employing the conventional voice agent function or the like is not necessarily good.

In order to solve the above problems, an information processing apparatus according to a preferred aspect of the present invention includes an acquisition unit that acquires content information regarding content, and a user input in a natural language for an application that processes the content, in the content information. And an interpreting unit for interpreting based on the information.

According to the present invention, the usability of the information processing device can be improved.

FIG. 1 is a block diagram illustrating an overall configuration of an information processing apparatus according to a first embodiment of the present invention. FIG. 4 is an explanatory diagram illustrating an example of content information. FIG. 9 is an explanatory diagram illustrating an example in which interpretation of a user input is uniquely specified. FIG. 11 is an explanatory diagram illustrating another example in which the interpretation of a user input is uniquely specified. FIG. 9 is an explanatory diagram illustrating an example in which interpretation of a user input is not uniquely specified; 3 is a flowchart illustrating an example of an operation of the information processing apparatus illustrated in FIG. It is a block diagram showing the whole information processing unit composition concerning a 2nd embodiment of the present invention. FIG. 8 is an explanatory diagram illustrating an example of an operation of the information processing device illustrated in FIG. 7. It is a block diagram showing the whole information processor concerning a 3rd embodiment of the present invention. FIG. 5 is an explanatory diagram illustrating an example of a relationship between content information and an output mode of response information. 10 is a flowchart illustrating an example of an operation of the information processing apparatus illustrated in FIG.

[1. First Embodiment]
FIG. 1 is a block diagram illustrating an overall configuration of an information processing apparatus 10 according to the first embodiment of the present invention. In the following description, a smartphone is assumed as the information processing device 10. However, any portable information processing device can be adopted as the information processing device 10, and may be, for example, a notebook computer, a wearable terminal, a tablet terminal, or the like.

As illustrated in FIG. 1, the information processing device 10 is realized by a computer system including a processing device 100, a storage device 140, an input device 150, an output device 160, and a communication device 170. A plurality of elements of the information processing device 10 are mutually connected by a single or a plurality of buses. Note that the term “apparatus” in this specification may be replaced with another term such as a circuit, a device, or a unit. In addition, each of the plurality of elements of the information processing device 10 may be configured by a single device or a plurality of devices. Alternatively, some elements of the information processing device 10 may be omitted.

The processing device 100 is a processor that controls the entire information processing device 10, and is configured by, for example, a single chip or a plurality of chips. The processing device 100 includes, for example, a central processing unit (CPU: Central Processing Unit) including an interface with a peripheral device, an arithmetic device, a register, and the like. Note that some or all of the functions of the processing device 100 are implemented by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be realized. The processing device 100 executes various types of processing in parallel or sequentially.

The processing device 100 functions as the agent unit 110 by, for example, reading out and executing the control program PR from the storage device 140. The agent unit 110 interprets a user input, which is a user input in a natural language, and executes a process according to the user input. The user input is, for example, an instruction or a question from the user in a natural language. The method of user input (the method of user input in natural language) is not particularly limited as long as the information processing apparatus 10 can convert the content of the user input into text or the like and interpret it. For example, the user input method corresponds to input by voice, text, or the like.

The acquiring unit 112, the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118 shown in the agent unit 110 of FIG. 1 are examples of functional blocks of the agent unit 110. That is, the information processing apparatus 10 includes the obtaining unit 112, the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118.

(4) The acquisition unit 112 acquires content information on content. For example, the acquisition unit 112 acquires content information on content being processed by an application that is in a state of receiving a user input. In the following, the content that is being processed by the application that is in a state of accepting user input is also referred to as valid content. For example, the acquisition unit 112 specifies an application that is being executed by the information processing apparatus 10, and specifies valid content based on the name of the application or a file that is being processed by the application. Then, the obtaining unit 112 obtains content information on valid content.

For example, when the user is watching a movie using the information processing device 10, the acquisition unit 112 specifies the movie as valid content. Further, for example, when the user has created a transmission message of a mail using the information processing device 10, the acquisition unit 112 specifies the mail as valid content. Then, the obtaining unit 112 obtains content information on valid content. In this specification, a mail is a type of content because a user refers to the mail. The content information has one or a plurality of parameters determined according to the type of the content. For example, when the effective content is a TV (television) program, the obtaining unit 112 obtains a plurality of parameters including the title (program information) of the TV program. An example of the content information will be described with reference to FIG.

The interpretation unit 114 interprets a user input to an application that processes valid content based on the content information. For example, the interpretation unit 114 interprets the content of the user input based on the parameters included in the content information. For example, when the valid content is a TV program and the title, which is one of a plurality of parameters related to the TV program, indicates a baseball broadcast, when the user asks “What other games?” The user input “other games?” Is interpreted as a search for a game result of another baseball or a search for a progress of another baseball.

The control command issuing unit 116 issues a control command according to a user input based on the result of interpretation by the user input interpreting unit 114. For example, when the user input “other games?” Is interpreted by the search and interpretation unit 114 for searching for a result of another baseball game or for searching for the progress of another baseball game, the control command issuing unit 116 performs data broadcasting. A control command for retrieving the result of another baseball game or the progress of another baseball from information and the like included in the above is issued. By issuing a control command for searching for the result of another baseball game or the course of another baseball, the result of another baseball game or the course of another baseball is searched, and the search result is acquired by the response information generation unit 118. You.

The response information generation unit 118 generates response information to the user input based on the result of the interpretation by the user input interpretation unit 114. The response information to the user input is, for example, information indicating that the instruction from the user has been received, information indicating the execution result of the process in response to the instruction from the user, information indicating the answer to the user's question, and the like. For example, if the user input “other games?” Is interpreted by the search and interpretation unit 114 for searching for a game result of another baseball or searching for the progress of another baseball, the response information generating unit 118 Response information indicating a search result of a baseball game result or a search result of another baseball game in progress is generated. As a result, for example, based on the response information, the game result of another baseball or the progress of another baseball is displayed as text on a display 162 described later.

When the interpretation result of the user input interpretation unit 114 includes a plurality of interpretations, the response information generation unit 118 generates response information for confirming which of the plurality of interpretations is applicable to the content of the user input. . That is, when the interpretation of the user input is not uniquely specified, the response information generation unit 118 generates response information for asking the user about the content of the user input. An example in which the interpretation of the user input is not uniquely specified will be described with reference to FIG.

The storage device 140 is a recording medium that can be read by the processing device 100, and stores a plurality of programs including the control program PR executed by the processing device 100, and various data used by the processing device 100. The storage device 140 may be constituted by at least one of a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory).

The input device 150 is an input device that receives an external input. For example, the input device 150 includes a microphone 152 that receives a voice input operation and an operation unit 154 that receives an operation by a user. The input device 150 transfers the user input received by the microphone 152 or the operation unit 154 to the agent unit 110.

The microphone 152 receives, for example, a user input such as an instruction or a question from the user by voice. The operation unit 154 is a device (for example, a keyboard, a mouse, a switch, a button, and the like) for inputting information used by the information processing device 10 to the processing device 100, and receives an instruction from a user or a user input such as a question. Accept. Specifically, operation unit 154 receives an operation for inputting codes such as numerals and characters to processing device 100 and an operation for selecting an icon displayed on display 162. For example, a touch panel that detects contact with the display surface of the display 162 is suitable as the operation unit 154. Note that the operation unit 154 may include a plurality of operators that can be operated by the user. Further, the input device 150 may include a sensor that detects a movement or the like of the information processing device 10 itself.

The output device 160 is an output device that performs output to the outside. For example, the output device 160 includes a display 162, a speaker 164, and a light emitting unit 166. The display 162 is an example of a display device, and displays various images under the control of the processing device 100. For example, the display 162 displays an image such as text or an icon indicating response information. As the display 162, for example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used.

The speaker 164 outputs various sounds under the control of the processing device 100. For example, the speaker 164 outputs sound such as voice or music indicating response information.

The light emitting unit 166 has a light emitting element such as an LED (Light Emitting Diode), and emits various lights under the control of the processing device 100. For example, the processing device 100 turns on or blinks the light emitting unit 166 according to the content of the response information.

The communication device 170 is a device that communicates with another device via a mobile communication network or a network such as the Internet. The communication device 170 is also described as, for example, a network device, a network controller, a network card, or a communication module. Next, an example of content information will be described with reference to FIG.

FIG. 2 is an explanatory diagram showing an example of content information. The content information has, for example, a content type and one or more parameters determined according to the content type. In the example shown in FIG. 2, a plurality of parameters are determined according to the type of content. By using a plurality of parameters for interpreting the contents of the user input, it is possible to more efficiently specify the user's instruction even when an ambiguous instruction is issued from the user, as compared with the case where one parameter is used.

If the type of content is a movie or a TV program, the parameters are, for example, information indicating the title of the movie or the TV program, the presence or absence of subtitles, the window size, the presence / absence of audio mute, the presence / absence of earphone connection, and the like. For example, when the parameter indicating the presence or absence of subtitles is set to a first value (for example, the value “1”), it indicates that subtitles are present, and a second value (for example, value If “0” is set, it indicates that there is no subtitle. The window size indicates, for example, whether a window displaying a movie or a TV program is full screen display or reduced display.

If the content type is music, the parameter is information indicating, for example, the title of the song, whether or not to display lyrics, the window size, whether or not an earphone is connected, and the like. The window size indicates, for example, whether a window displaying operation buttons for operating music reproduction or the like is displayed in full screen or reduced.

If the content type is e-mail, the parameters are, for example, information indicating the type of active window, the window size, the presence / absence of audio muting, the presence / absence of earphone connection, and the like. The types of windows in the active state are, for example, a received mail list screen, a received mail display screen, a transmitted mail creation screen, and the like. The window size indicates, for example, whether the active window is a full screen display or a reduced display.

If the content type is a map, the parameters are information indicating, for example, the display magnification of the map, the window size, the presence / absence of audio mute, the presence / absence of earphone connection, and the like. The window size indicates, for example, whether the window displaying the map is full screen display or reduced display.

When the type of content is an action game such as a battle game, the parameters are information indicating, for example, the title of the game, the type of the game screen, the window size, the presence / absence of audio mute, the presence / absence of earphone connection, and the like. . The types of game screens include, for example, a scene during a fight, a scene for selecting an item, and a screen for displaying a game result. The window size indicates, for example, whether the window displaying the game is full screen display or reduced display.

If the type of content is a music game, the parameters are information indicating, for example, the title of the game, the window size, the presence or absence of the connection of the earphone, and the like. The window size indicates, for example, whether the window displaying the game is full screen display or reduced display.

{Note that the type of content and the parameters determined according to the type of content are not limited to the example shown in FIG. For example, when the type of content is music, the parameter may be one piece of information indicating whether or not the earphone is connected. That is, the number of parameters included in the content information may be one.

FIG. 3 is an explanatory diagram showing an example in which the interpretation of the user input is uniquely specified. FIG. 3 shows an example of the operation of the information processing apparatus 10 when the user instructs by voice to “increase” while watching a movie using the information processing apparatus 10. When the valid content is a movie, “increase”, which is a user input, has two meanings: increasing the screen and increasing the volume. In the example illustrated in FIG. 3, it is assumed that the parameters included in the content information regarding the movie indicate audio mute, unconnected earphones, reduced display, and the like. For this reason, in the example illustrated in FIG. 3, the interpretation unit 114 uniquely interprets “enlarge”, which is a user input, as enlarging the screen.

For example, since the user is viewing a movie using the information processing apparatus 10, the acquisition unit 112 specifies the movie as valid content, and obtains a plurality of parameters including the title of the movie (for example, as illustrated in FIG. 2). Parameter). Then, the interpreting unit 114 uniquely interprets “enlarge”, which is a user input, as increasing the screen based on the parameters and the like acquired by the acquiring unit 112.

Since the interpretation result of the interpretation unit 114 is to enlarge the screen, the response information generation unit 118 generates response information indicating that the screen is enlarged as response information to a user input. As a result, for example, a text stating "Make full screen" is displayed on display 162.

{Circle around (2)} Since the interpretation result of the interpretation unit 114 is to enlarge the screen, the control command issuing unit 116 issues a control command for displaying the movie on the full screen. As a result, the movie is displayed on the entire screen of the display 162.

Note that the user may utter an instruction or the like after calling the information processing apparatus 10 with a predetermined word. For example, the input device 150 of the information processing device 10 may receive, as a user input, a voice following a call for a predetermined word. In this case, the information processing device 10 can easily determine whether or not the word uttered by the user is an input to the information processing device 10 by detecting the presence or absence of a call for a predetermined word.

ユ ー ザ In addition, the user input is not limited to the voice input, and may be, for example, text. For example, when the user inputs “I want to go to Shibuya” by text via the operation unit 154 while watching a movie, the information processing apparatus 10 searches for a route from the current position to Shibuya, and displays the search result in text. 162 may be displayed. Also in FIG. 4 and subsequent figures, the operation of the information processing apparatus 10 is described with an example where the user input is a voice, but the user input is not limited to the voice input.

FIG. 4 is an explanatory diagram showing another example in which the interpretation of the user input is uniquely specified. FIG. 4 shows an example of the operation of the information processing apparatus 10 when the user gives an instruction to increase the volume while listening to a movie using the information processing apparatus 10. In the example illustrated in FIG. 4, it is assumed that the parameters included in the content information regarding the movie indicate audio output (no mute), connection of earphones, and full-screen display. For this reason, in the example illustrated in FIG. 4, the interpretation unit 114 uniquely interprets “increase”, which is a user input, as increasing the volume.

For example, since the user is viewing a movie using the information processing apparatus 10, the acquisition unit 112 specifies the movie as valid content, and obtains a plurality of parameters including the title of the movie (for example, as illustrated in FIG. 2). Parameter). Then, the interpreting unit 114 uniquely interprets “increase”, which is a user input, as increasing the volume based on the parameters and the like acquired by the acquiring unit 112.

Since the interpretation result of the interpretation unit 114 is to increase the volume, the response information generation unit 118 generates response information indicating that the volume is increased as response information to a user input. As a result, for example, text indicating “increase the volume” is displayed on the display 162.

{Circle around (2)} Since the interpretation result of interpreting section 114 is to increase the volume, control command issuing section 116 issues a control command to increase the volume. As a result, the volume of the movie being played by the information processing device 10 increases.

The response information when the interpretation of the user input is uniquely specified is not limited to the examples illustrated in FIGS. 3 and 4. For example, the information processing apparatus 10 may display a text “OK” on the display 162 as a response to the user input.

As described in FIGS. 3 and 4, even when the content of the user input has a plurality of meanings, the information processing apparatus 10 can uniquely specify the interpretation of the user input based on the content information on the valid content. For this reason, usability of the information processing apparatus 10 can be improved. An example in which the interpretation of the user input cannot be uniquely specified even by referring to the content information regarding the valid content will be described with reference to FIG.

FIG. 5 is an explanatory diagram showing an example where the interpretation of the user input is not uniquely specified. FIG. 5 shows an example of the operation of the information processing device 10 when the user gives an instruction to increase the size while listening to a movie using the information processing device 10. In the example shown in FIG. 5, it is assumed that the parameters included in the content information regarding the movie indicate audio output (no mute), connection of earphones, and reduced display. For this reason, in the example illustrated in FIG. 5, the interpretation unit 114 interprets “increase”, which is a user input, in an ambiguous manner as increasing the screen or increasing the volume.

For example, since the user is viewing a movie using the information processing apparatus 10, the acquisition unit 112 specifies the movie as valid content, and obtains a plurality of parameters including the title of the movie (for example, as illustrated in FIG. 2). Parameter). Then, the interpreting unit 114 interprets “increase”, which is a user input, in an ambiguous manner as increasing the screen or increasing the volume based on the parameters and the like acquired by the acquiring unit 112.

Since the interpretation by the interpretation unit 114 of “increase” which is a user input includes a plurality of interpretations (enlarging the screen and increasing the volume), the response information generation unit 118 Generates response information asking the user whether the interpretation is applicable to the contents of the user input. For example, the response information generation unit 118 generates response information for specifying the content of the user input such as “Is the screen to be increased or the volume?”. As a result, a text stating “Is the screen to be increased or the volume?” Is displayed on the display 162.

In the example shown in FIG. 5, the user instructs “screen” and voice in response to the question “Is the screen to be increased or the volume?”. For this reason, the interpreting unit 114 uniquely interprets the first user input “enlarge” as enlarging the screen. Then, the response information generation unit 118 generates response information indicating that the user's instruction is to be executed. Further, the control command issuing unit 116 issues a control command for displaying a movie on the entire screen. As a result, for example, the text “OK” is displayed on the display 162, and the movie is displayed on the entire screen of the display 162.

Even when the interpretation of the user input by the interpretation unit 114 includes a plurality of interpretations, the information processing apparatus 10 uses the response information asking the user which of the plurality of interpretations is applicable to the content of the user input. Identify the interpretation of the input. For this reason, usability of the information processing apparatus 10 can be improved.

FIG. 6 is a flowchart showing an example of the operation of the information processing apparatus shown in FIG. The operation illustrated in FIG. 6 is an example of a control method of the information processing device 10.

In step S100, the processing device 100 determines whether or not there is a user input. For example, the processing device 100 determines whether the input device 150 has received a user input. When there is a user input, that is, when the input device 150 receives the user input, the operation of the information processing device 10 proceeds to step S110. On the other hand, when there is no user input, that is, when the input device 150 has not received the user input, the operation of the information processing device 10 returns to step S100.

That is, the information processing apparatus 10 waits for the execution of the processing in step S110 until the input device 150 receives a user input. In other words, when the input device 150 receives a user input (for example, a voice input such as “increase” in FIGS. 3, 4, and 5), the information processing device 10 executes the process of step S110.

In step S110, the processing device 100 functions as the acquisition unit 112 and specifies valid contents. In the examples illustrated in FIGS. 3 to 5, since the user is viewing a movie using the information processing device 10, the acquisition unit 112 specifies the movie as valid content. For example, when the user is viewing the mail using the information processing apparatus 10, the mail is specified as valid content, and when the user is viewing the map using the information processing apparatus 10, the map is valid content. Specified as Also, for example, when the user is playing an action game using the information processing device 10, the action game is specified as valid content, and when the user is playing a music game using the information processing device 10, the music game Is specified as valid content.

Next, in step S120, the processing device 100 functions as the obtaining unit 112, and obtains content information including one or more parameters indicating a valid content state. For example, when the valid content identified in step S110 is a movie, the acquisition unit 112 sets parameters indicating the title of the movie, the presence or absence of subtitles, the window size, the presence or absence of audio mute, and the presence or absence of earphone connection, respectively. , As parameters to be included in the content information.

Next, in step S130, the processing device 100 functions as the interpretation unit 114, and interprets the content of the user input based on the content information acquired in step S120. In the example illustrated in FIG. 3, the interpretation unit 114 uniquely interprets “enlarge”, which is a user input, based on the enlargement of the screen and the parameters included in the content information. Further, in the example shown in FIG. 4, the interpretation unit 114 uniquely interprets “increase”, which is a user input, based on increasing the volume, parameters included in the content information, and the like. In the example illustrated in FIG. 5, the interpreting unit 114 sets the user input “increase” in an ambiguous manner based on increasing the screen or increasing the volume and parameters included in the content information. To be interpreted.

Next, in step S140, the processing device 100 functions as the response information generation unit 118, and determines whether the content of the user input interpreted in step S130 is ambiguous. For example, the response information generation unit 118 determines whether the interpretation result of the user input by the interpretation unit 114 includes a plurality of interpretations.

In the examples shown in FIGS. 3 and 4, the interpretation result of the user input by the interpretation unit 114 indicates one interpretation, so that the content of the user input is uniquely specified. Therefore, in the examples shown in FIGS. 3 and 4, the response information generation unit 118 determines that the content of the user input is not ambiguous. Further, in the example shown in FIG. 5, since the interpretation result of the user input by the interpretation unit 114 includes a plurality of interpretations, the content of the user input is not uniquely specified. Therefore, in the example shown in FIG. 5, the response information generation unit 118 determines that the content of the user input is ambiguous.

Note that the determination as to whether the content of the user input interpreted in step S130 is ambiguous may be executed by a functional block other than the response information generation unit 118. For example, it may be determined whether the content of the user input interpreted by the interpreting unit 114 in step S130 is ambiguous. If the content of the user input is ambiguous, that is, if the interpretation result of the user input by the interpretation unit 114 includes a plurality of interpretations, the operation of the information processing apparatus 10 proceeds to step S142. On the other hand, when the content of the user input is not ambiguous, that is, when the content of the user input is uniquely specified, the operation of the information processing apparatus 10 proceeds to step S150.

In step S142, the processing device 100 functions as the response information generation unit 118, and generates response information for asking the user about the contents of the user input based on the interpretation result in step S130. Then, the information processing device 10 outputs the generated response information. In the example illustrated in FIG. 5, the response information generation unit 118 uses the interpretation result of “increase” as the user input (two interpretations of increasing the screen and increasing the volume). Response information for asking the user about the contents of the user input such as "Is the screen to be increased or the volume?" Then, the information processing apparatus 10 displays a text stating “Is the screen to be increased or the volume?” On the display 162.

Next, in step S144, the processing device 100 functions as the interpretation unit 114, and determines the interpretation of the content of the user input based on the response to the response information output in step S142. In the example illustrated in FIG. 5, the interpreting unit 114 receives the answer “Screen” from the user in response to the question “Do you want to increase the screen or volume?” The interpretation of the content of "enlarge" is determined to enlarge the screen. After the process of step S144 is performed, the operation of the information processing device 10 proceeds to step S150.

In step S150, the processing device 100 functions as the response information generation unit 118, and generates response information to the user input based on the interpretation result of the content of the user input. Then, the information processing device 10 outputs the generated response information. For example, if the content of the user input interpreted in step S130 is ambiguous, the response information generation unit 118 generates response information based on the interpretation of the content of the user input determined in step S144. Further, for example, when the content of the user input interpreted in step S130 is unique, the response information generation unit 118 generates response information according to the content of the user input interpreted in step S130.

In the example illustrated in FIG. 3, the user input “enlarge” is uniquely interpreted as enlarging the screen, so that the response information generation unit 118 outputs the response information indicating that the screen is enlarged to the user. Generate as response information to input. Then, the information processing device 10 displays a text stating “Change to full screen” on the display 162 based on the generated response information. In the example illustrated in FIG. 4, “increase”, which is a user input, is uniquely interpreted as increasing the volume, so that the response information generation unit 118 generates response information indicating that the volume is increased. , As response information to a user input. Then, the information processing device 10 displays a text indicating “increase the volume” on the display 162 based on the generated response information.

In the example illustrated in FIG. 5, the interpretation of “enlarge” as the user input is determined to enlarge the screen based on the response to the response information asking the user for the content of the user input. Generates response information indicating that the user's instruction is to be executed. Then, the information processing device 10 displays a text “OK” on the display 162 based on the generated response information.

Next, in step S160, the processing device 100 functions as the control command issuing unit 116, and generates a control command corresponding to the user input based on the interpretation result of the content of the user input. In the examples shown in FIGS. 3 and 5, since the interpretation result of the content of “enlarge”, which is a user input, is to enlarge the screen, the control command issuing unit 116 performs control to display the movie on the full screen. Issue a command. As a result, the movie is displayed on the entire screen of the display 162. In the example shown in FIG. 4, since the interpretation result of the content of "increase", which is a user input, is to increase the volume, the control command issuing unit 116 issues a control command to increase the volume. As a result, the volume of the movie being played by the information processing device 10 increases.

Next, in step S170, the processing device 100 functions as the response information generation unit 118, and generates response information to the user input based on the execution result of the control command issued in step S160. Then, the information processing device 10 outputs the generated response information. In the examples shown in FIGS. 3, 4, and 5, the response to the user input ends by executing the control command issued in step S160. However, the end of the response to the user input is not limited to the execution of the control command issued in step S160. For example, when the content of the user input is a route search to a destination, the response to the user input ends by outputting the result of the route search.

For example, if the content of the user input is a route search to the destination, a control command for executing the route search to the destination is issued in step S160, and the response information generation unit 118 determines the route to the destination. Is generated based on the result of the route search. Then, the information processing apparatus 10 outputs the route to the destination by one or both of text and voice, and ends the response to the user input.

The operation of the information processing device 10 is not limited to the example illustrated in FIG. For example, when the interpretation of the content of the user input is not determined in step S144, a series of processes in steps S142 and S144 may be repeated until the interpretation of the content of the user input is determined. Also, for example, one of the processes of steps S150 and S170 may be omitted according to the content of the user input.

As described above, in the first embodiment, the information processing device 10 interprets a user input (an input of a user in a natural language) to an application that processes content based on the content information. And an interpreting unit 114. The information processing device 10 interprets the content of the user input based on the content information on the valid content. Therefore, for example, when an ambiguous instruction is issued from the user, it is possible to reduce the possibility that the user's instruction is not specified, and to reduce the execution of a process different from the user's intention. As a result, the usability of the information processing device 10 can be improved.

The information processing apparatus 10 further includes a control command issuing unit 116 that issues a control command according to a user input based on the result of interpretation by the user input interpreting unit 114. For example, even when an ambiguous instruction is issued from the user, the user's instruction is uniquely interpreted by the interpreting unit 114 based on the content information, so that a control command for a process different from the user's intention is issued. Can be reduced.

The information processing apparatus 10 also includes a response information generation unit 118 that generates response information to a user input based on the result of interpretation by the user input interpretation unit 114. For example, even when an ambiguous instruction is issued from the user, the instruction of the user is uniquely interpreted by the interpretation unit 114 based on the content information, so that response information is generated for an instruction different from the user's intention. Can be reduced.

When the interpretation result of the user input interpretation unit 114 includes a plurality of interpretations, the response information generation unit 118 generates response information asking the user which of the plurality of interpretations is applicable to the content of the user input. I do. For example, when the content of the user input cannot be uniquely specified even by using the content information on the valid content, the information processing apparatus 10 specifies the content of the user input by using the response information asking the user about the content of the user input. it can.

[2. Second Embodiment]
The main difference between the second embodiment and the above-described first embodiment is that the agent unit 110a shown in FIG. 7 has an acquisition unit 112a instead of the acquisition unit 112 shown in FIG.

FIG. 7 is a block diagram showing the overall configuration of the information processing apparatus 10 according to the second embodiment of the present invention. Elements that are the same as or similar to the elements described in FIGS. 1 to 6 are denoted by the same reference numerals, and detailed description is omitted.

情報 処理 The information processing device 10 shown in FIG. 7 has the same configuration as the information processing device 10 shown in FIG. For example, the information processing device 10 is realized by a computer system including a processing device 100, a storage device 140, an input device 150, an output device 160, and a communication device 170. A plurality of elements of the information processing device 10 are mutually connected by a single or a plurality of buses. In addition, each of the plurality of elements of the information processing device 10 may be configured by a single device or a plurality of devices. Alternatively, some elements of the information processing device 10 may be omitted.

The processing device 100 shown in FIG. 7 is the same as or similar to the processing device 100 shown in FIG. 1 except that the control device PR is executed instead of the control program PR shown in FIG. For example, the processing device 100 functions as the agent unit 110a by reading and executing the control program PRa from the storage device 140.

The agent unit 110a interprets a user input, which is a user input in a natural language, and executes processing according to the user input, similarly to the agent unit 110 illustrated in FIG. Note that the acquiring unit 112a, the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118 shown in the agent unit 110a of FIG. 7 are examples of functional blocks of the agent unit 110a. That is, the information processing apparatus 10 includes the obtaining unit 112a, the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118. The interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118 illustrated in FIG. 7 are the same as the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118 illustrated in FIG. Therefore, FIG. 7 illustrates the acquisition unit 112a.

When a plurality of windows are displayed on the display 162, the acquisition unit 112a specifies a content corresponding to an active window that receives a user input from among the plurality of windows. Then, the obtaining unit 112a obtains content information on the content corresponding to the window in the active state. For example, when the window displaying the map is in an active state among a plurality of windows displayed on the display 162, the acquiring unit 112a specifies the display of the map as valid content. Then, the obtaining unit 112a obtains content information on valid content.

FIG. 8 is an explanatory diagram showing an example of the operation of the information processing apparatus 10 shown in FIG. FIG. 8 shows an example of the operation of the information processing apparatus 10 when two windows WD (WD10 and WD20) are displayed on the display 162. A movie is displayed in window WD10, and a mail is displayed in window WD20. The dark shaded upper portion in the window WD in FIG. 8 indicates the active window WD for receiving a user's input.

In the state C1, the acquiring unit 112a specifies the window WD10 as the active window WD that receives a user input. Then, the acquisition unit 112a specifies the movie reproduced in the window WD10 as valid content. Therefore, the obtaining unit 112a obtains content information on a movie. In the state C2, the acquisition unit 112a specifies the window WD20 as the active window WD that receives a user input. Then, the acquisition unit 112a specifies the mail displayed in the window WD20 as valid content. Therefore, the obtaining unit 112a obtains the content information regarding the mail.

As described above, also in the second embodiment, the same effects as in the first embodiment can be obtained. In the second embodiment, when a plurality of windows WD are displayed on the display 162, the acquisition unit 112a validates the content corresponding to the active window WD that receives a user input among the plurality of windows WD. Identify as content. For this reason, even when a plurality of windows WD are displayed on the display 162, the information processing apparatus 10 can interpret the user input based on the content information on the valid content. Therefore, even when a plurality of windows WD are displayed on the display 162, the usability of the information processing apparatus 10 can be improved.

[3. Third Embodiment]
The main difference between the third embodiment and the above-described first embodiment is that the output mode of the response information is determined based on the content information on the valid content.

FIG. 9 is a block diagram showing the overall configuration of the information processing apparatus 10 according to the third embodiment of the present invention. Elements that are the same as or similar to the elements described with reference to FIGS. 1 to 8 are given the same reference numerals, and detailed descriptions thereof will be omitted.

情報 処理 The information processing apparatus 10 shown in FIG. 9 has the same configuration as the information processing apparatus 10 shown in FIG. 1 except that an output device 160A is provided instead of the output device 160 shown in FIG. For example, the information processing device 10 is realized by a computer system including the processing device 100, the storage device 140, the input device 150, the output device 160A, and the communication device 170. A plurality of elements of the information processing device 10 are mutually connected by a single or a plurality of buses. In addition, each of the plurality of elements of the information processing device 10 may be configured by a single device or a plurality of devices. Alternatively, some elements of the information processing device 10 may be omitted.

The output device 160A has the same configuration as the output device 160 shown in FIG. 1 except that the output device 160A has the vibration generating unit 168. That is, the output device 160A includes the display 162, the speaker 164, the light emitting unit 166, and the vibration generating unit 168. The vibration generator 168 is, for example, a vibrator, and vibrates under the control of the processing device 100. Specifically, the processing device 100 vibrates the information processing device 10 by vibrating the vibration generating unit 168 according to the content of the response information. The processing device 100 may set the pattern of the vibration according to the content of the response information to a pattern different from the pattern of the vibration indicating the incoming call or the like.

The processing device 100 shown in FIG. 9 is the same as or similar to the processing device 100 shown in FIG. 1 except that the control device PR shown in FIG. 1 is executed instead of the control program PR. For example, the processing device 100 functions as the agent unit 110b, the display data generation unit 120, and the sound data generation unit 130 by reading and executing the control program PRb from the storage device 140.

The agent unit 110b interprets a user input, which is a user input in a natural language, and executes a process according to the user input, similarly to the agent unit 110 illustrated in FIG. Note that the acquiring unit 112, the interpreting unit 114, the control command issuing unit 116, the response information generating unit 118, and the output mode determining unit 119 shown in the agent unit 110b of FIG. 9 are examples of functional blocks of the agent unit 110b. That is, the information processing apparatus 10 includes the obtaining unit 112, the interpreting unit 114, the control command issuing unit 116, the response information generating unit 118, and the output mode determining unit 119. The acquiring unit 112, the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118 illustrated in FIG. 9 are the same as the interpreting unit 114, the control command issuing unit 116, and the response information generating unit 118 illustrated in FIG. Therefore, FIG. 9 illustrates the output mode determination unit 119, the display data generation unit 120, and the sound data generation unit 130.

(4) The output mode determining unit 119 determines the output mode of the response information based on the content information on the valid content. For example, the output mode determination unit 119 selects an output mode of response information from output mode candidates including a plurality of output modes based on the content information. The output mode candidates include, for example, an output mode in which response information is output as an image, an output mode in which response information is output as sound, an output mode in which response information is output as vibration, and an output mode in which light according to the content of the response information is output. Output modes.

The output mode of outputting the response information as an image may include, for example, an output mode of displaying the content of the response information in text and an output mode of displaying an icon corresponding to the content of the response information. The output mode of outputting the response information as a sound includes, for example, an output mode in which a text indicating the content of the response information is read out, and music such as a melody, harmony, rhythm (or tempo), and timbre that can identify the content of the response information. And an output mode of outputting the target element.

When the output mode of the response information is determined by the output mode determining unit 119 to be an output mode in which the response information is output as an image, the display data generation unit 120 generates display data such as a text or an icon indicating the content of the response information. . Then, the display data generation unit 120 transfers the generated display data to the display 162.

When the output mode of the response information is determined by the output mode determining unit 119 to be an output mode in which the response information is output as a sound, the sound data generating unit 130 generates sound data indicating the content of the response information. The sound data is, for example, sound data that reads out text indicating the content of the response information, or sound data that includes a musical element capable of identifying the content of the response information. The sound data generation unit 130 transfers the generated sound data to the speaker 164.

The function block of the agent unit 110b is not limited to the example shown in FIG. For example, the agent unit 110b may include the acquisition unit 112a illustrated in FIG. Next, an example of the relationship between the content information and the output mode of the response information will be described with reference to FIG.

FIG. 10 is an explanatory diagram illustrating an example of a relationship between content information and an output mode of response information. Note that the relationship between the content information and the output mode of the response information is not limited to the example illustrated in FIG. In FIG. 10, information indicated by one of the plurality of parameters is extracted and described. For example, when the type of content is a movie or a TV program, parameter information indicating the presence or absence of subtitles is described, and when the type of content is mail, parameter information indicating a window size is described.

For example, when the type of content is a movie or a TV program and there is no subtitle, text is selected as the output mode of the response information. The information processing device 10 can prevent the sound of a movie or the like from being difficult to hear by responding to the user input in a text display.

If the type of content is a movie or a TV program and there is a caption, a text in a font different from the caption is selected as the output mode of the response information. The information processing apparatus 10 displays the content of the response information in a typeface different from the subtitle of the movie, thereby easily distinguishing whether the text displayed on the display 162 indicates the content of the response information or the subtitle of the movie. Can be. In addition, the information processing apparatus 10 displays the content of the response information at a position on the display 162 that does not overlap the subtitles, thereby preventing the subtitles of the movie from being difficult to see.

(4) When the content type is mail and full-screen display, voice is selected as the output mode of the response information. The information processing apparatus 10 can prevent the text and the like of the mail from being difficult to read by responding to the user input by voice. For example, when the output mode of the response information is text, if the text indicating the content of the response information is displayed over the text of the mail, the text of the mail becomes difficult to read.

(4) When the content type is mail and reduced display, both voice and text are selected as output modes of the response information. By responding to the user input with both voice and text, the information processing apparatus 10 can reliably convey the contents of the response information to the user, as compared with the case of responding only with voice. Further, the information processing apparatus 10 displays the content of the response information in an area different from the display area of the mail on the display 162, so that it is possible to prevent the text of the mail from being difficult to read. For example, when the text indicating the content of the response information is displayed in the display area of the mail, if the text indicating the content of the response information is displayed over the text of the mail, the text of the mail becomes difficult to read.

(4) When the content type is a map and full-screen display, voice is selected as the output mode of the response information. In this case, it is possible to prevent the displayed map from being difficult to see. For example, when the output mode of the response information is text, if the text indicating the content of the response information is displayed over the map, the displayed map becomes difficult to see. When the type of content is a map and reduced display, both voice and text are selected as output modes of the response information. In this case, the contents of the response information can be reliably transmitted to the user, as compared with the case of responding only by voice. The information processing apparatus 10 displays the contents of the response information in an area different from the display area of the map on the display 162, thereby preventing the map being displayed from being difficult to see.

If the type of content is an action game and full screen display, voice is selected as the output mode of the response information. In this case, it is possible to prevent the game screen or the like from being difficult to see, and it is possible to prevent the progress of the action game from being hindered. For example, when the output mode of the response information is text, if the text indicating the content of the response information is displayed over the game screen or the like, the game screen or the like becomes difficult to see. When the type of content is an action game and the display is reduced, both voice and text are selected as the output mode of the response information. In this case, the contents of the response information can be transmitted to the user more reliably than in the case of responding only by voice. The information processing apparatus 10 displays the content of the response information in an area different from the display area of the action game on the display 162, thereby preventing the game screen and the like from being difficult to see.

If the content type is a music game and full screen display, text is selected as the response information output mode. Note that, even when the content type is a music game and the display is reduced, text is selected as the output mode of the response information. The information processing apparatus 10 can prevent the sound of the game from being difficult to hear by responding to the user's input with a text, and can suppress the trouble in the progress of the music game. For example, when the output mode of the response information is voice, if the voice that conveys the content of the response information to the user overlaps with the sound of the game, it becomes difficult to hear the content of the response information and the sound of the game.

FIG. 11 is a flowchart showing an example of the operation of the information processing apparatus 10 shown in FIG. The operation illustrated in FIG. 11 is an example of a control method of the information processing device 10. The operation illustrated in FIG. 11 is the same as or similar to the operation illustrated in FIG. 6 except that the process of step S132 is added to the operation illustrated in FIG. Therefore, in FIG. 11, the operation of the information processing apparatus 10 will be described focusing on the processing of step S132. The process of step S132 is executed, for example, after the process of step S130 is executed.

In step S132, the processing device 100 functions as the output mode determining unit 119, and determines the output mode of the response information based on the content information acquired in step S120. For example, in steps S142, S150, and S170, response information is output in the output mode determined in the process of step S132. After the processing of step S132 is performed, the processing of step S140 is performed.

The operation of the information processing device 10 is not limited to the example illustrated in FIG. For example, if the process of step S132 is performed after the process of step S120 is performed, it may be performed before the process of step S130 is performed. Further, for example, the output mode determining unit 119 may determine the output mode of the response information in consideration of one or both of the content of the user input and the content of the response information in addition to the content information. That is, the output mode determining unit 119 may determine the output mode of the response information based on the content information and the content of the user input, or may determine the response mode based on the content information, the content of the user input, and the content of the response information. The output mode of the information may be determined. For example, when the content of the user input is a highly urgent request or the like, or when the content of the response information is something that the user wants to be surely recognized (for example, a highly urgent content), the response information May be selected as both text and voice.

Further, for example, when the response information is information that conveys a simple content such as that the user has accepted the instruction, the information processing apparatus 10 may transmit the response information to the user by vibrating the vibration generating unit 168. . Alternatively, when the response information is information that conveys simple contents, the information processing apparatus 10 may convey the response information to the user by turning on or off a light emitting unit 166 such as an LED, or may output a short sound to the speaker 164. May output the response information to the user.

As described above, also in the third embodiment, the same effects as in the first embodiment can be obtained. In the third embodiment, the information processing device 10 includes an output mode determination unit 119 that determines the output mode of the response information based on the content information. For example, the information processing apparatus 10 can change the output mode of the response information to the user input in accordance with the content information regarding the valid content. Therefore, as described with reference to FIG. 10, usability of the information processing apparatus 10 can be improved.

[4. Modification]
The present invention is not limited to the embodiment exemplified above. Specific modifications will be described below. Two or more aspects arbitrarily selected from the following examples may be combined.

[First Modification]
In the above-described second embodiment, an example has been described in which the content corresponding to the window WD in the active state is specified as the valid content among the plurality of windows WD displayed on the display 162, but the valid content is in the active state. The content is not limited to the content corresponding to the window WD. For example, when the operation on the window WD in the active state is not performed for a predetermined time or more, the acquiring unit 112 determines that the predetermined highest priority content among the plurality of contents respectively corresponding to the plurality of windows WD is the valid content. May be specified. For example, in the state C2 in FIG. 8, when the user continues to watch the movie displayed in the window WD10 without performing an operation on the mail for a predetermined time or more after creating and transmitting the outgoing mail, When the priority of the movie is higher than the priority of the mail, the acquisition unit 112a may specify the movie as valid content instead of the mail in the active state.

[Second Modification]
In each of the above-described first to third embodiments, the output devices 160 and 160A have the light-emitting unit 166, but the light-emitting unit 166 outputs light corresponding to the content of the response information. In a case where the output mode is not included in the output mode candidates, the light emitting unit 166 may be omitted from the output devices 160 and 160A. In addition, the output device 160 may include the vibration generation unit 168 in a case where the output mode of outputting the response information by vibration is included in the output mode candidate.

[Third Modification]
The information processing device 10 may include an auxiliary storage device. The auxiliary storage device is a recording medium readable by the processing device 100, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (eg, a compact disk, a digital versatile disk). , A Blu-ray (registered trademark) disk), a smart card, a flash memory (for example, a card, a stick, a key drive), a floppy (registered trademark) disk, and a magnetic strip. The auxiliary storage device may be called a storage.

[5. Others]
(1) In the above-described embodiment, the storage device 140 is a recording medium readable by the processing device 100, such as a ROM and a RAM. However, a flexible disk, a magneto-optical disk (for example, a compact disk, Disk, Blu-ray (registered trademark) disk, smart card, flash memory device (eg, card, stick, key drive), CD-ROM (Compact Disc-ROM), register, removable disk, hard disk, floppy (registered trademark) ) Disks, magnetic strips, databases, servers and other suitable storage media. Further, the program may be transmitted from a network via a telecommunication line. Further, the program may be transmitted from a communication network via a telecommunication line.

(2) The above-described embodiments are based on LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), A system using IEEE@802.16 (WiMAX (registered trademark)), IEEE@802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), and other appropriate systems, and a next-generation system extended based on these systems May be applied. A plurality of systems may be combined (for example, a combination of at least one of LTE and LTE-A with 5G) and applied.

用語 Note that terms described in the present disclosure and terms necessary for understanding the present disclosure may be replaced with terms having the same or similar meaning. For example, the signal may be a message.

(3) In the above-described embodiment, input and output information and the like may be stored in a specific place (for example, a memory), or may be managed using a management table. Information that is input and output can be overwritten, updated, or added. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

(4) In the above-described embodiment, the determination may be made based on a value (0 or 1) represented by one bit, or may be performed based on a Boolean value (Boolean: true or false). , May be performed by comparing numerical values (for example, comparison with a predetermined value).

(5) The order of the processing procedures, sequences, flowcharts, and the like exemplified in the above-described embodiment may be changed as long as there is no contradiction. For example, for the methods described in this disclosure, elements of various steps are presented in an exemplary order, and are not limited to the specific order presented.

(6) Each function illustrated in FIGS. 1, 7 and 9 is realized by an arbitrary combination of at least one of hardware and software. In addition, a method of implementing each functional block is not particularly limited. That is, each functional block may be realized using one device physically or logically coupled, or directly or indirectly (for example, two or more devices physically or logically separated from each other). , Wired, wireless, etc.), and may be implemented using these multiple devices. The functional block may be realized by combining one device or the plurality of devices with software.

(7) In the program exemplified in the above-described embodiment, the software is an instruction, an instruction set, a code, regardless of whether it is called software, firmware, middleware, microcode, a hardware description language, or another name. Should be interpreted broadly to mean code segment, program code, program, subprogram, software module, application, software application, software package, routine, subroutine, object, executable, thread of execution, procedure, function, etc. .

ソ フ ト ウ ェ ア Also, software, instructions, information, and the like may be transmitted and received via a transmission medium. For example, if the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and wireless technology (infrared, microwave, etc.), the website, When transmitted from a server or other remote source, at least one of these wired and / or wireless technologies is included within the definition of a transmission medium.

(8) In the above embodiments, the terms “system” and “network” are used interchangeably.

(9) The information, parameters, and the like described in the present disclosure may be represented using an absolute value, may be represented using a relative value from a predetermined value, or may be represented using another corresponding information. May be expressed as The names used for the parameters described above are not limiting in any way. Further, equations and the like using these parameters may differ from those explicitly disclosed in the present disclosure.

(10) In the embodiments described above, the terms "connected," "coupled," or any variation thereof, refer to a direct or indirect connection between two or more elements. Any connection or combination is meant and may include the presence of one or more intermediate elements between two elements "connected" or "coupled" to each other. The coupling or connection between the elements may be physical, logical, or a combination thereof. For example, “connection” may be read as “access”. As used in this disclosure, two elements may be implemented using at least one of one or more wires, cables, and printed electrical connections, and as some non-limiting and non-exhaustive examples, in the radio frequency domain. , Can be considered "connected" or "coupled" to each other using electromagnetic energy having wavelengths in the microwave and optical (both visible and invisible) regions, and the like.

(11) In the above-described embodiment, the description “based on” does not mean “based only on” unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

(12) The terms "determining" and "determining" as used in the present disclosure may encompass a wide variety of operations. `` Judgment '', `` decision '', for example, judgment (judging), calculation (calculating), calculation (computing), processing (processing), derivation (deriving), investigating (investigating), searching (looking up, search, inquiry) (E.g., searching in a table, database, or another data structure), ascertaining may be considered "determined", "determined", and the like. Also, “determining” and “deciding” include receiving (eg, receiving information), transmitting (eg, transmitting information), input (input), output (output), and access. (accessing) (for example, accessing data in a memory) may be regarded as “determined” or “determined”. In addition, `` judgment '' and `` decision '' means that resolving, selecting, selecting, establishing, establishing, comparing, etc. are considered as `` judgment '' and `` decided ''. May be included. In other words, “judgment” and “decision” may include deeming any operation as “judgment” and “determined”. “Judgment (determination)” may be read as “assuming”, “expecting”, “considering”, or the like.

(13) In the embodiments described above, where “include”, “including” and variations thereof are used, these terms are used in the same manner as the term “comprising” It is intended to be comprehensive. Further, the term "or" as used in the present disclosure is not intended to be an exclusive or.

(14) In the present disclosure, when articles are added by translation, for example, a, an, and the in English, the present disclosure may include that the noun following these articles is plural. Good.

(15) Each aspect / embodiment described in the present disclosure may be used alone, may be used in combination, or may be used by switching with execution. Further, the notification of the predetermined information (for example, the notification of “X”) is not limited to being explicitly performed, and is performed implicitly (for example, not performing the notification of the predetermined information). Is also good.

Although the present disclosure has been described in detail above, it is obvious to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented as modified and changed aspects without departing from the spirit and scope of the present disclosure defined by the description of the claims. Therefore, the description of the present disclosure is intended for illustrative purposes, and has no restrictive meaning to the present disclosure.

DESCRIPTION OF SYMBOLS 10 ... Information processing apparatus, 100 ... Processing apparatus, 110, 110a, 110b ... Agent part, 112, 112a ... Acquisition part, 114 ... Interpretation part, 116 ... Control command issuing part, 118 ... Response information generation part, 119 ... Output mode Determination unit, 120: display data generation unit, 130: sound data generation unit, 140: storage device, 150: input device, 152: microphone, 154: operation unit, 160, 160A: output device, 162: display, 164: speaker 166: Light-emitting unit, 168: Vibration generator, 170: Communication device, WD10, WD20: Window.

Claims (7)

1. An acquisition unit for acquiring content information on the content,
   An interpretation unit that interprets a user input in a natural language for an application that processes the content based on the content information;
   An information processing apparatus comprising:
2. A control command issuing unit that issues a control command according to the user input based on an interpretation result of the user input by the interpretation unit,
   The information processing apparatus according to claim 1, wherein:
3. Response information to the user input, comprising a response information generation unit that generates based on the interpretation result of the user input by the interpretation unit,
   The information processing apparatus according to claim 1, wherein:
4. The response information generation unit, when the interpretation result of the user input by the interpretation unit includes a plurality of interpretations, the response information to ask the user which of the plurality of interpretations are applicable to the content of the user input Produces
   The information processing apparatus according to claim 3, wherein:
5. An output mode determining unit that determines an output mode of the response information based on the content information,
   The information processing apparatus according to claim 3, wherein:
6. When a plurality of windows are displayed on the display device, the obtaining unit obtains the content information on the content corresponding to an active window that receives a user input, among the plurality of windows,
   The information processing apparatus according to any one of claims 1 to 5, wherein:
7. The content information has a plurality of parameters determined according to the type of the content,
   The information processing apparatus according to any one of claims 1 to 6, wherein:

Images