WO2017130474A1

WO2017130474A1 - Information processing device, information processing method, and program

Info

Publication number: WO2017130474A1
Application number: PCT/JP2016/080485
Authority: WO
Inventors: 真一河野; 東山　恵祐; 伸樹古江; 圭祐齊藤; 佐藤　大輔; 亮介三谷; 美和市川
Original assignee: ソニー株式会社
Priority date: 2016-01-25
Filing date: 2016-10-14
Publication date: 2017-08-03
Also published as: US20190019511A1; JPWO2017130474A1; JP6841239B2; EP3410432A1; US11120063B2; EP3410432A4

Abstract

Provided is an information processing device provided with a processing unit which, on the basis of acquired information indicating the importance of summarization, performs summarization processing in which utterance content indicated by speech information based on an utterance of a user is summarized.

Description

Information processing apparatus, information processing method, and program

The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Technology for summarizing electronic documents has been developed. As a technique for summarizing an electronic document and adding a tag indicating copyright information to the created summary sentence, for example, a technique described in Patent Document 1 below can be cited.

JP 2001-167114 A

When a person who speaks (hereinafter referred to as “speaker”) speaks, it is difficult to speak only what the speaker wants to convey.

Therefore, for example, assuming that communication can be made by utterances, “In addition to the content that the speaker wants to convey, content other than the content that the speaker wants to convey (ie, unnecessary content) is transmitted to the other party that is communicating. "There are many. Therefore, when communication is achieved by utterance, it may happen that “the other party who communicates takes time to understand what the speaker wants to convey”.

Also, assuming that the content of the utterance is translated into another language, for example, “In addition to the content that the speaker wants to convey, content other than the content that the speaker wants to convey is caused by the speaker” Thus, “translation takes time” and “translation result not intended by the speaker” may occur.

Here, “speaking only what the speaker wants to communicate, such as“ the person with whom it takes communication takes time to understand what the speaker wants to communicate ”or“ it takes time to translate ” As a method of further reducing the possibility of the occurrence of an “event due to difficulty of”, a method of simplifying the content of the speaker's utterance can be considered.

This disclosure proposes a new and improved information processing apparatus, information processing method, and program capable of summarizing the content of an utterance.

According to the present disclosure, there is provided an information processing apparatus including a processing unit that performs a summarization process for summarizing the content of an utterance indicated by voice information based on a user's utterance based on information indicating a weight related to the acquired summary.

Moreover, according to this indication, based on the information which shows the weight regarding the acquired summary, it is performed by the information processing apparatus which has a step which performs the summarization process which summarizes the content of the speech which the audio | voice information based on a user's speech shows An information processing method is provided.

Further, according to the present disclosure, there is provided a program for causing a computer to implement a function of performing a summarization process for summarizing the content of utterances indicated by voice information based on a user's utterances based on information indicating weights related to the acquired summaries. Provided.

れば According to the present disclosure, the content of utterances can be summarized.

Note that the above effects are not necessarily limited, and any of the effects shown in the present specification, or other effects that can be grasped from the present specification, together with or in place of the above effects. May be played.

It is explanatory drawing for demonstrating an example of the use case to which the information processing method which concerns on this embodiment is applied. It is explanatory drawing for demonstrating an example of the use case to which the information processing method which concerns on this embodiment is applied. It is explanatory drawing for demonstrating an example of the use case to which the information processing method which concerns on this embodiment is applied. It is explanatory drawing for demonstrating an example of the use case to which the information processing method which concerns on this embodiment is applied. It is explanatory drawing for demonstrating an example of the use case to which the information processing method which concerns on this embodiment is applied. It is explanatory drawing which shows an example of the table for setting the weight regarding the summary which concerns on this embodiment. It is explanatory drawing which shows an example of the table for setting the weight regarding the summary which concerns on this embodiment. It is explanatory drawing which shows an example of the table for setting the weight regarding the summary which concerns on this embodiment. It is explanatory drawing for demonstrating an example of the summary process which concerns on a 1st information processing method. It is explanatory drawing for demonstrating an example of the summary process which concerns on a 1st information processing method. It is explanatory drawing for demonstrating an example of the summary process which concerns on a 1st information processing method. It is explanatory drawing which shows an example of the notification by the visual method implement | achieved by the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is explanatory drawing for demonstrating an example of the notification control process which concerns on a 2nd information processing method. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a flowchart which shows an example of the process which concerns on the information processing method which concerns on this embodiment. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on this embodiment. It is explanatory drawing which shows an example of the hardware constitutions of the information processing apparatus which concerns on this embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

In the following, description will be given in the following order.
1. 1. Information processing method according to this embodiment 2. Information processing apparatus according to this embodiment Program according to this embodiment

(Information processing method according to this embodiment)
First, an information processing method according to the present embodiment will be described. Below, the case where the information processing apparatus concerning this embodiment performs the process concerning the information processing method concerning this embodiment is mentioned as an example.

In the following, the information processing method according to the present embodiment will be described by dividing it into a first information processing method and a second information processing method. In the following description, a case where the same information processing apparatus performs both the processing related to the first information processing method and the processing related to the second information processing method will be mainly described. The information processing apparatus that performs the process according to the above may be different from the information processing apparatus that performs the process according to the second information processing method.

In the following, a person who is a target of processing related to the information processing method according to the present embodiment is indicated as “user”. As a user according to the present embodiment, for example, “speaker (or a person who can be a speaker)” (when a first information processing method described later is performed) or “operator of an operation device related to notification” ( And a second information processing method to be described later).

[1] Outline of information processing method according to this embodiment [1-1] Outline of first information processing method As described above, “because it is difficult to utter only the content that the speaker wants to convey As a method for further reducing the possibility of the “event to occur”, a method for simplifying the content of the utterance of the speaker can be considered.

Therefore, the information processing apparatus according to the present embodiment performs processing for summarizing the content of the utterance (hereinafter referred to as “summarization processing”) as processing related to the first information processing method. The information processing apparatus according to the present embodiment summarizes the content of the utterance indicated by the voice information based on the user's utterance based on the information indicating the weight related to the acquired summary. As the summary according to the present embodiment, for example, the content of an utterance is selected based on the weight related to the summary, or a part is extracted from the content of the utterance based on the weight related to the summary.

The information indicating the weight related to the summary includes, for example, data indicating the weight related to the summary, which is stored in a table (or database, hereinafter the same applies) for setting the weight related to the summary described later. Further, the information indicating the weight regarding the summary may be data indicating that the weight regarding the summary is relatively large or small. The information indicating the weight related to the summary is acquired by referring to a table for setting the weight related to the summary described later, for example.

Here, the voice information according to the present embodiment is voice data including voice based on the utterance of the speaker. The voice information according to the present embodiment is generated when a voice input device such as a microphone picks up voice based on the utterance of the speaker. Also, the audio information according to the present embodiment may be information obtained by converting an analog signal generated according to the audio picked up by the audio input device into a digital signal by an AD (Analog-to-Digital) converter. The voice input device (or the voice input device and the AD converter) may be included in the information processing apparatus according to the present embodiment, or may be a device external to the information processing apparatus according to the present embodiment. There may be.

The content of the utterance indicated by the voice information includes, for example, a character string indicated by text data (hereinafter, referred to as “voice text information”) obtained as a result of arbitrary voice recognition processing performed on the voice information. It is done. The information processing apparatus according to the present embodiment recognizes the character string indicated by the voice text information as the content of the utterance indicated by the voice information, and summarizes the character string indicated by the voice text information.

Here, the voice recognition processing for the voice information may be performed by the information processing apparatus according to the present embodiment, or may be performed by an external device of the information processing apparatus according to the present embodiment. When the information processing apparatus according to the present embodiment performs the speech recognition process, the information processing apparatus according to the present embodiment indicates the character indicated by the speech text information obtained as a result of performing the speech recognition process on the acquired speech information. Summarize the column. When the external device of the information processing apparatus according to the present embodiment performs speech recognition processing, the information processing apparatus according to the present embodiment summarizes the character string indicated by the speech text information acquired from the external device.

Further, in the information processing apparatus or the external apparatus according to the present embodiment, the voice recognition process may be repeatedly performed, for example, periodically / non-periodically, or at a predetermined trigger such as a timing when the voice information is acquired. It may be done accordingly. In the information processing apparatus or the external apparatus according to the present embodiment, the voice recognition process may be performed when a predetermined operation such as a voice recognition start operation related to the summary is performed, for example.

The weight related to the summary according to the present embodiment is an index for extracting more important words (in other words, words that the speaker will want to convey) from the content of the utterance indicated by the voice information. is there. Based on the weight related to the summary according to the present embodiment, the content of the utterance indicated by the voice information is summarized, so that more important words corresponding to the weight related to the summary are included in the content of the summarized utterance. .

The weight related to the summary according to the present embodiment is at least one of audio information, information about the user, information about the application, information about the environment, and information about the device (1 or 2 of these) as shown below, for example. Is set based on the above.

Here, the information on the user according to the present embodiment includes, for example, at least one of user status information indicating the user status and user operation information based on the user operation.

As the user state, for example, an action taken by the user (including an operation such as a gesture), a state of emotion of the user, and the like can be mentioned. For example, the user state is one or more of user biometric information obtained from an arbitrary biosensor, a detection result of a motion sensor such as a velocity sensor or an angular velocity sensor, and a captured image captured by an imaging device. It is estimated by an arbitrary action estimation process or an arbitrary emotion estimation process used. The processing related to the estimation of the user state may be performed by the information processing apparatus according to the present embodiment, or may be performed by an external device of the information processing apparatus according to the present embodiment. In addition, examples of user operations include various operations such as a speech recognition start operation related to summarization and an operation for starting a predetermined application.

Also, the information about the application indicates, for example, the execution state of the application.

In addition, the information regarding the environment indicates, for example, a situation around the user (or a situation where the user is placed). Examples of the environmental information include data indicating the level of noise around the user. The level of noise around the user is specified by threshold processing using one or two or more thresholds for level classification, for example, by extracting non-speech from voice information generated by a microphone. The processing related to the acquisition of information related to the environment as described above may be performed by the information processing apparatus according to the present embodiment, or may be performed by an external apparatus of the information processing apparatus according to the present embodiment.

Also, the information about the device indicates, for example, one or both of the device type and the device state. Examples of the state of the device include a processing load of a processor included in the device.

A specific example of processing related to the setting of weights related to summarization will be described later.

The content of the utterance indicated by the voice information is summarized by performing the summarization process according to the first information processing method. Therefore, the content of the utterance of the speaker indicated by the voice information can be simplified.

In the summarization process according to the first information processing method, for example, the content of the utterance is summarized based on the weights related to the summary set as described above, so that more important words corresponding to the weights related to the summary are summarized. Included in the content of the utterance.

Therefore, when the summarization process according to the first information processing method is performed, “the communication partner needs time to understand what the speaker wants to convey” or “translation takes time”. It is possible to obtain a summary result that can further reduce the possibility of occurrence of “an event caused by difficulty in speaking only the content that the speaker wants to convey”.

[1-2] Outline of Second Information Processing Method By performing the summarization process according to the first information processing method, it is possible to obtain the content of the utterance indicated by the summarized voice information.

The information processing apparatus according to the present embodiment performs processing for controlling notification of notification content (hereinafter referred to as “notification control processing”) based on summary information as processing related to the second information processing method.

Here, the summary information according to the present embodiment indicates the content of the summarized utterance corresponding to the voice information based on the utterance of the first user. The summary information is obtained, for example, by performing summary processing according to the first information processing method. Note that the content of the summarized utterance indicated by the summary information is not limited to the above, and may be summarized by any method capable of summarizing the utterance content indicated by the speech information based on the user's utterance. Good. Hereinafter, a case will be described as an example where the summary information indicates the content of the summarized utterance obtained by performing the summary processing according to the first information processing method.

And the information processing apparatus according to the present embodiment controls the notification of the notification content to the second user. Here, the notification content for the second user may be, for example, the content of the summarized utterance indicated by the summary information, or the content of the notification that is different from the content of the summarized utterance, or the summary The content of the summarized utterance indicated by the summary information, such as the translated content of the utterance, may not be used. Further, the first user according to the present embodiment and the second user according to the present embodiment may be different or the same. As an example of the case where the first user and the second user are different, there is a case where the first user is a speaker and the second user is a communication partner. Moreover, the case where a 1st user and a 2nd user are the same as an example as a case where a 1st user and a 2nd user are the same speaker is mentioned.

The information processing apparatus according to the present embodiment causes notification contents to be notified by one or both of notification by a visual method and notification by an auditory method, for example.

In the case where the notification is performed by a visual method, the information processing apparatus according to the present embodiment, for example, causes the notification content to be displayed on the display screen of the display device. The information processing apparatus according to the present embodiment transmits, for example, a display control signal including display data corresponding to the notification content and a display command to the display device, so that the notification content is displayed on the display screen of the display device. Display.

Here, as a display screen for displaying the notification content, for example, a display device that constitutes a display unit (described later) included in the information processing apparatus according to the present embodiment, or an external device of the information processing apparatus according to the present embodiment. Display devices. When the display screen for displaying the notification content is an external display device, the information processing apparatus according to the present embodiment includes, for example, a communication unit (described later) included in the information processing apparatus according to the present embodiment, or the present embodiment. The display control signal is transmitted to an external display device to a communication device external to the information processing apparatus.

Further, in the case where notification by an audible method is performed, the information processing apparatus according to the present embodiment outputs, for example, the notification content from a sound output device such as a speaker as sound (music may be included). To let you know. For example, the information processing apparatus according to the present embodiment transmits the audio output control signal including the audio data indicating the audio corresponding to the notification content and the audio output command to the audio output device. , Voice output from the audio output device.

Here, the audio output device that outputs the notification content by audio may be, for example, an audio output device included in the information processing apparatus according to the present embodiment, or an audio output outside the information processing apparatus according to the present embodiment. It may be a device. When the audio output device that outputs the notification content by voice is an external audio output device, the information processing apparatus according to the present embodiment includes, for example, a communication unit (described later) included in the information processing apparatus according to the present embodiment, or The audio output control signal is transmitted to the external audio output device to the communication device external to the information processing apparatus according to the present embodiment.

Note that the notification method of the notification content in the information processing apparatus according to the present embodiment is not limited to one or both of the notification method using the visual method and the notification method using the auditory method. For example, the information processing apparatus according to the present embodiment can also notify a break in the notification content by a tactile notification method, for example, by vibrating a vibration device.

By performing the notification control process according to the second information processing method, for example, the notification content based on the summarized utterance content obtained by the summarization process according to the first information processing method is notified.

Here, as described above, the content of the summarized utterance obtained by the summary processing according to the first information processing method is “because it is difficult to utter only the content that the speaker wants to convey” This is a summary result that can further reduce the possibility of the “event to occur”.

Therefore, when the summarization process according to the second information processing method is performed, the notification content is notified, so that “the communication partner needs time to understand the content that the speaker wants to convey”. It is possible to further reduce the possibility of occurrence of “an event caused by difficulty in speaking only the content that the speaker wants to convey” such as “it takes time for translation”.

[1-3] Other Processes Related to Information Processing Method According to the Present Embodiment The processes related to the information processing method according to the present embodiment include the summary process related to the first information processing method and the second information described above. It is not restricted to the notification control process which concerns on a processing method.

For example, the process related to the information processing method according to the present embodiment includes a process (hereinafter referred to as “translation process”) for translating the content of the utterance summarized by the summary process according to the first information processing method into another language. .) May be further included. By performing the translation process, the content of the summarized utterance is translated from the first language corresponding to the speech information based on the utterance into a second language different from the first language. Hereinafter, the content of the translated summary utterance obtained by performing the translation process is referred to as “translation result”.

Here, the translation processing according to the present embodiment may be performed as part of the processing according to the first information processing method, or may be performed as part of the processing according to the second information processing method.

In addition, in the processing related to the information processing method according to the present embodiment, one or both of the result of the summary processing according to the first information processing method and the result of the translation processing according to the present embodiment may be arbitrarily set. A recording control process for recording on the recording medium may be further included.

In the recording control process, for example, “one or both of the result of the summary process according to the first information processing method and the result of the translation process according to the embodiment” and “position information corresponding to the user” Information related to the user such as the user's biometric information obtained from an arbitrary biosensor (described later) may be associated and recorded as a log. By storing the log as described above in the recording medium, for example, “the user looks back on the record of the trip etc. afterwards” is realized.

[2] An example of a use case to which the information processing method according to the present embodiment is applied Next, while describing an example of a use case to which the information processing method according to the present embodiment is applied, the information processing according to the present embodiment An example of processing according to the method will be described. Hereinafter, as a use case to which the information processing method according to the present embodiment is applied, the information processing method according to the present embodiment is applied to “conversation support” (including a case where translation is performed as described later). The case where it will be described.

The use case to which the information processing method according to this embodiment is applied is not limited to “conversation support”. For example, the information processing method according to the present embodiment can be applied to any use case in which the content of the utterance indicated by the audio information can be summarized as described below.
・ "Meeting with meeting letters" realized by summarizing the utterances indicated by the voice information indicating the voice of the meeting, generated by an IC (Integrated Circuit) recorder, etc.
・ "Program telop automatic creation" realized by summarizing the content of utterances indicated by audio information indicating audio in a television program
・ One or both of “automatic conference telop creation” and “come to conference text” realized by summarizing the utterances indicated by audio information indicating audio in a video conference

1 to 5 are explanatory diagrams for explaining an example of a use case to which the information processing method according to this embodiment is applied.

In FIG. 1, FIG. 2, and FIG. 5, the person indicated by “U1” corresponds to the user according to the present embodiment. 2 and 5, the person indicated by “U2” corresponds to the partner with whom the user U1 communicates. In the following, the person indicated by “U1” in FIGS. 1, 2 and 5 is indicated as “user U1”, and the person indicated by “U2” in FIGS. 2 and 5 is indicated as “communication partner U2”. Show. In the following, a case where the native language of the communication partner U2 is Japanese is taken as an example.

FIG. 1, FIG. 2, and FIG. 5 show an example in which the user U1 is wearing an eyewear type device having a display screen. In addition, an audio input device such as a microphone, an audio output device such as a speaker, and an imaging device are connected to the eyewear type apparatus worn by the user U1 shown in FIGS. Yes.

Moreover, in an example of the use case shown below, as the information processing apparatus according to the present embodiment, for example, a wearable apparatus used by being worn on the body of the user U1, such as the eyewear type apparatus shown in FIG. Examples include a communication device such as a smartphone and a computer such as a server. Note that the information processing apparatus according to the present embodiment is not limited to the example described above. An application example of the information processing apparatus according to the present embodiment will be described later.

Hereinafter, an example of a use case to which the information processing method according to the present embodiment is applied will be described with reference to FIGS. 1 to 5 as appropriate.

Suppose an English-speaking user U1 arrives at a Japanese airport by plane.

(A) Example of processing related to setting of weight related to summary The information processing apparatus according to the present embodiment sets a weight related to a summary, for example, by using a table for setting a weight related to the summary. Here, the table for setting the weight regarding the summary may be stored in a storage unit (described later) included in the information processing apparatus according to the present embodiment, or may be stored outside the information processing apparatus according to the present embodiment. It may be stored in a recording medium. The information processing apparatus according to the present embodiment uses, for example, a table for setting a weight related to summarization by appropriately referring to a storage unit (described later) or an external recording medium.

In addition, the information processing apparatus according to the present embodiment can set the weight related to the summary by determining the weight related to the summary using an arbitrary algorithm for determining the weight related to the summary, for example.

6 to 8 are explanatory diagrams showing examples of tables for setting the weights related to the summary according to the present embodiment.

FIG. 6 shows an example of a table for specifying the weight related to the summary, and shows an example of the table weighted for each type of weight related to the summary for each registered vocabulary. Here, in FIG. 6, among the combinations of the weight type and vocabulary regarding the summary, the combination indicated by the value “1” corresponds to the weighted combination. In FIG. 6, among the combinations of the weight type and vocabulary regarding the summary, the combination indicated by the value “0” corresponds to the combination that is not weighted.

7 and 8 show examples of tables for specifying the types of weights related to the summary. FIG. 7 shows an example of a table in which schedule contents specified from the state of the schedule application (or schedule contents estimated from the state of the schedule application) and weight types related to the summary are associated with each other. FIG. 8 shows an example of a table in which user behavior (an example of a user state) is associated with a summary weight type.

The information processing apparatus according to the present embodiment includes, for example, a table for specifying the type of weight related to the summary as shown in FIGS. 7 and 8 and a table for specifying the weight related to the summary as shown in FIG. By using both as tables for setting the weight for the summary, the weight for the summary is set.

In addition, the example of the table for specifying the kind of weight regarding the summary which concerns on this embodiment is not restricted to the example shown to FIG. 7, FIG. 8, and the example of the table for specifying the weight regarding a summary is, It goes without saying that the present invention is not limited to the example shown in FIG. Moreover, the table for setting the weight regarding the summary which concerns on this embodiment may be provided for every languages, such as Japanese, English, Chinese, for example.

In addition, the information processing apparatus according to the present embodiment determines the type of weight related to summarization based on at least one of, for example, audio information, information about a user, information about an application, information about an environment, and information about a device. In this case, it is possible to set the weight for the summary using only the table for specifying the weight for the summary as shown in FIG.

The information processing apparatus according to the present embodiment, for example, as illustrated in FIG. 6 based on a recognition result based on at least one of audio information, information about a user, information about an application, information about an environment, and information about a device. From the table for specifying the weight for the summary, the type of the weight for the summary is determined by selecting the type of the weight for the summary related to the recognition result. Then, the information processing apparatus according to the present embodiment refers to, for example, a table for specifying the weight related to the summary as illustrated in FIG. , A weight is set for the vocabulary corresponding to the combination indicated by the value “1”.

As a specific example, the information processing apparatus according to the present embodiment sets a weight related to the summary by performing, for example, any one of the following processes (a-1) to (a-5).

Note that examples relating to the setting of weights related to summarization are not limited to the examples shown in (a-1) to (a-5) below. For example, the information processing apparatus according to the present embodiment can set a weight related to summarization according to a language recognized based on voice information. Examples of weight settings for summarization according to language include, for example, “if the language recognized based on speech information is Japanese, increase the verb weight” or “recognize based on speech information. If the language used is English, increase the weight of the nouns ”. In addition, the information processing apparatus according to the present embodiment relates to, for example, a weight related to the summary according to the situation around the user indicated by the information related to the environment, and a summary corresponding to the content (for example, device type) indicated by the information related to the device Each weight may be set.

(A-1) First example of setting of weight related to summary: an example of setting of weight related to summary based on user status indicated by user status information included in information related to user For example, user U1 is a device such as a smartphone When the schedule application is activated to confirm the destination, the information processing apparatus according to the present embodiment recognizes that the user U1 is moving with respect to the destination. Then, the information processing apparatus according to the present embodiment sets the weight related to the summary corresponding to the recognition result by referring to the table for setting the weight related to the summary.

As a specific example, the information processing apparatus according to the present embodiment determines the type of weight related to the summary illustrated in FIG. 8 based on the recognition result that the user U1 obtained as described above is moving to the destination. From the table for specifying, “time” corresponding to the action “moving” is specified as the type of weight related to the summary. Then, the information processing apparatus according to the present embodiment refers to the table for specifying the weight related to the summary illustrated in FIG. 6, and the value of the combination of the weight type and the vocabulary related to the specified summary is “ A weight is set for the vocabulary corresponding to the combination indicated by 1 ″. When the table for specifying the weight related to the summary shown in FIG. 6 is used, weights are set for the vocabulary “AM”, “when”,.

Further, when the user U1 operates a device such as a smartphone and starts a game application, the information processing apparatus according to the present embodiment recognizes that the user U1 is playing a game. Then, the information processing apparatus according to the present embodiment sets the weight related to the summary corresponding to the recognition result by referring to the table for setting the weight related to the summary.

For example, the information processing apparatus according to the present embodiment, based on the recognition result that the user U1 obtained as described above is playing a game, from the table for specifying the type of weight related to the summary illustrated in FIG. The “game term” corresponding to the action “in game” is specified as the type of weight related to the summary. The information processing apparatus according to the present embodiment refers to the table for specifying the weights related to the summary illustrated in FIG. 6, and the value of the combination of the determined weight type and vocabulary related to the summary is “ A weight is set for the vocabulary corresponding to the combination indicated by 1 ″.

In addition, the information processing apparatus according to the present embodiment is included in the table for specifying the weight related to the summary illustrated in FIG. 6 based on the recognition result that the user U1 obtained as described above is in the game. It is also possible to determine the type of weight related to the summary related to the recognition result such as “game term” as the type of weight related to the summary. The information processing apparatus according to the present embodiment refers to the table for specifying the weights related to the summary illustrated in FIG. 6, and the value of the combination of the determined weight type and vocabulary related to the summary is “ A weight is set for the vocabulary corresponding to the combination indicated by 1 ″.

In addition, the information processing apparatus according to the present embodiment recognizes the state of the user U1 estimated based on the detection result of a motion sensor such as an acceleration sensor or an angular velocity sensor provided in an apparatus such as a smartphone used by the user U1, for example. It is also possible to set a weight for the summary based on the result.

For example, when the recognition result that the user U1 is eating is obtained based on the detection result of the motion sensor, the action “meal” is selected from the table for specifying the type of weight related to the summary illustrated in FIG. The “cooking” corresponding to is specified as a weight type related to the summary. The information processing apparatus according to the present embodiment refers to the table for specifying the weights related to the summary illustrated in FIG. 6, and the value of the combination of the determined weight type and vocabulary related to the summary is “ A weight is set for the vocabulary corresponding to the combination indicated by 1 ″.

(A-2) Second example of weight setting for summarization: an example of setting weight for summarization based on voice information The information processing apparatus according to the present embodiment sets weights for summarization based on voice information.

The information processing apparatus according to the present embodiment determines the type of weight related to the summary, for example, as follows based on the audio information.
When the average frequency band of the voice indicated by the voice information is, for example, 300 to 550 [Hz]: “Male” is determined as the type of weight related to the summary.
When the average frequency band of the voice indicated by the voice information is, for example, 400 to 700 [Hz]: “Women” is determined as the type of weight regarding the summary.
-When the sound pressure and volume of the sound indicated by the sound information are equal to or higher than the set first threshold value, or when the sound pressure and sound volume of the sound indicated by the sound information are larger than the first threshold value: One or both of “anger” and “joy” is determined as the type.
-When the sound pressure and volume of the sound indicated by the sound information are less than or equal to the set second threshold value, or when the sound pressure and sound volume of the sound indicated by the sound information are smaller than the second threshold value: , “Sadness”, “discomfort”, “pain”, and “anxiety” are determined.
When the pitch (sound pitch) or utterance speed (phoneme amount per unit time) indicated by the voice information is greater than the set third threshold, or the voice pitch indicated by the voice information or When the speaking rate is equal to or higher than the third threshold value: “excitement” is determined as the type of weight related to the summary.
When the voice pitch or speech speed indicated by the voice information is smaller than the set fourth threshold or when the voice pitch or speech speed indicated by the voice information is equal to or lower than the fourth threshold: “Silence” is determined as the weight type.

As the first threshold value, for example, a fixed value such as 72 [dB] can be cited. Moreover, as said 2nd threshold value, fixed values, such as 54 [dB], are mentioned, for example. Note that the first threshold value and the second threshold value may change dynamically depending on the distance between a user such as the user U1 and a communication partner such as the communication partner U2. . As an example of dynamically changing the first threshold value and the second threshold value, for example, “every time the distance approaches 0.5 [m], the threshold value is increased by 6 [dB] and moved away by 0.5 [m]. Every 6 [dB] lowering ”. The distance may be estimated by, for example, arbitrary image processing on a captured image captured by an imaging device, or may be acquired by a distance sensor. When the distance is estimated, the process related to the distance estimation may be performed by the information processing apparatus according to the present embodiment or may be performed by an external apparatus of the information processing apparatus according to the present embodiment.

Further, the third threshold value and the fourth threshold value may be fixed values set in advance, or may be variable values that can be changed based on a user operation or the like.

Note that the type of weight related to the summary determined based on the audio information is not limited to the example shown above.

For example, the emotion (eg, anger, joy, sadness) is estimated based on one or both of the number of mora obtained from the voice information and the location of the accent, and the type of weight related to the summary corresponding to the estimated emotion is determined. It is possible to set. When setting the type of weight related to the summary corresponding to the estimated emotion, the information processing apparatus according to the present embodiment, for example, the rate of change of the fundamental frequency obtained from the speech information, the rate of change of the sound, Based on the rate of change or the like, the strength of the weight related to emotion may be changed.

Here, as in the first example shown in (a-1) above, the information processing apparatus according to the present embodiment provides a table for specifying the types of weights related to summarization as shown in FIGS. The type of weight related to the summary may be determined using the table, or the weight related to the summary may be determined using only the table for specifying the weight related to the summary as shown in FIG.

When the weight related to the summary is determined, the information processing apparatus according to the present embodiment specifies the weight related to the summary as illustrated in FIG. 6 as in the first example illustrated in (a-1), for example. With reference to the table, weighting is set for the vocabulary corresponding to the combination whose value is indicated by “1” among the combinations of the weight type and the vocabulary regarding the specified summary.

(A-3) Third example of weight setting for summarization: an example of setting weight for summarization based on the execution state of the application indicated by the information regarding the application The information processing apparatus according to this embodiment is based on the execution state of the application. To set the weight for the summary.

For example, when the user U1 operates a device such as a smartphone to start a schedule application and confirms a destination, the information processing apparatus according to the present embodiment is based on the execution state of the schedule application, as shown in FIG. From the table for specifying the type of weight related to the summary shown in (2), “time” and “location” corresponding to the schedule content “place move (biz)” are specified as the types of weight related to the summary. Then, the information processing apparatus according to the present embodiment refers to the table for specifying the weight related to the summary illustrated in FIG. 6, and the value of the combination of the weight type and the vocabulary related to the specified summary is “ A weight is set for the vocabulary corresponding to the combination indicated by 1 ″. When the table for specifying the weights related to the summary shown in FIG. 6 is used, weights are set for the vocabularies “AM”, “Shibuya”, “when”, “where”,. .

Also, the information processing apparatus according to the present embodiment can determine the type of weight related to the summary based on the properties of the application being executed, for example, and set the weight related to the summary as described below.
When the map application is executed: “Time”, “Location”, “Person name”, etc. are determined as the types of weights related to the summary.
When the transfer guidance application is executed: “Time”, “Place”, “Train”, etc. are determined as the types of weights related to the summary.
When an application for smoothly advancing questions for hearing about Japan is being executed: “Question”, “Japan”, etc. are determined as the types of weights related to summarization.

(A-4) Fourth example of setting of weight related to summary: an example of setting of weight related to summary based on user operation indicated by user operation information included in information related to user The information processing apparatus according to this embodiment includes: A summary weight is set based on the user's operation.

The information processing apparatus according to the present embodiment, for example, uses the type of weight related to the summary selected by the operation of selecting the type of weight related to the summary (an example of the user's operation) to set the weight related to the summary. Decide as the type.

Further, the information processing apparatus according to the present embodiment, for example, when a predetermined operation such as a speech recognition start operation related to the summary is performed, weights related to the summary that are associated in advance with the predetermined operation. The type may be set automatically. As an example, when a speech recognition start operation related to a summary is performed, “question” or the like is determined as the type of weight related to the summary.

(A-5) Fifth Example of Weight Setting for Summarization The information processing apparatus according to the present embodiment relates to summarization by combining two or more of (a-1) to (a-4) above. It is possible to set a weight.

(B) An example of summary processing according to the first information processing method For example, when the user U1 wants to throw away trash at the station while moving toward the destination, there is no trash box at the station. Assume a case where the communication partner U2 is inquired in English about the reason why there is not (FIGS. 1 and 2).

Here, when the communication partner U2 cannot sufficiently understand English, it is highly likely that the communication partner U2 cannot fully understand the contents visited by the user U1.

Therefore, the information processing apparatus according to the present embodiment performs a summarization process according to the first information processing method, and, for example, an utterance indicated by voice information generated by a microphone connected to the eyewear type apparatus illustrated in FIG. The contents of are summarized. As described above, the information processing apparatus according to the present embodiment summarizes a character string indicated by voice text information based on voice information, for example.

More specifically, the information processing apparatus according to the present embodiment uses, for example, the content of the utterance by the objective function using the weight related to the summary set by the processing shown in (a) as shown in Equation 1 below. To summarize.

Here, “W” shown in Equation 1 is a weight related to the summary. Further, “a _i ” shown in Equation 1 is a parameter for adjusting the contribution rate of each weight related to the summary, and takes a real number from 0 to 1, for example. Also, it is shown in Equation 1 _{"z yi"} is, if included phrase _{y i} "1" indicates, unless contains phrase _{y i} is a binary variable indicating the "0".

Note that the information processing apparatus according to the present embodiment is not limited to the method using the objective function using the weight related to the summarization shown in Equation 1, but can summarize the content of the utterance using the set weight related to the summarization. Any possible method can be used.

FIG. 3 shows an example of the result of the summary process according to the first information processing method. FIG. 3A shows an example of the content of an utterance before it is summarized. 3B shows an example of the content of the summarized utterance, and C of FIG. 3 shows another example of the content of the summarized utterance.

As shown in FIG. 3B, the content of the utterance is summarized, so that the content of the utterance is simplified before the content is summarized. Therefore, even if the communication partner U2 cannot fully understand English by summarizing the content of the utterance as shown in FIG. 3B, the communication partner U2 is visited by the user U1. It is possible to increase the possibility of understanding the content.

In addition, C in FIG. 3 indicates that “the information processing apparatus according to the present embodiment further performs morphological analysis on the summary result illustrated in B in FIG. 3 and combines the morphemes based on the result of the morphological analysis. An example in which the divided text obtained by dividing the summary result shown in B of FIG. 3 is used as the content of the summarized utterance ”is shown.

For example, when the language of the character string indicated by the speech text information corresponding to the content of the utterance is Japanese, the information processing apparatus according to this embodiment includes the main part of speech (noun, verb, adjective, adverb) Split text is generated in units that combine morphemes other than. For example, when the language of the character string indicated by the speech text information corresponding to the content of the utterance is English, the information processing apparatus according to the present embodiment further sets 5W1H as the divided text.

As the content of the utterance is summarized as shown in FIG. 3C, the content of the utterance is simplified more than the summary result shown in B of FIG. Therefore, even if the communication partner U2 cannot fully understand English by summarizing the content of the utterance as shown in FIG. 3C, the communication partner U2 is visited by the user U1. The possibility that the contents can be understood can be further increased as compared with the case of obtaining the summary result shown in FIG.

(C) Example of Translation Process The information processing apparatus according to this embodiment may further translate the content of the utterance summarized by the summary process shown in (b) above into another language, for example. As described above, the information processing apparatus according to the present embodiment translates the first language corresponding to the utterance into a second language different from the first language.

The information processing apparatus according to the present embodiment specifies, for example, the position where the user U1 exists, and the language of the character string indicated by the speech text information corresponding to the content of the utterance is different from the official language at the specified position. In some cases, the content of the summarized utterance is translated into the official language. The position where the user U1 exists is acquired from, for example, a wearable device worn by the user U1 such as the eyewear-type device shown in FIG. 1 or a communication device such as a smartphone possessed by the user U1. Specified based on location information. As the location information, for example, data indicating the detection result of a device capable of specifying a location such as a GNSS (Global Navigation Satellite System) device (or estimation of a device capable of estimating the location by an arbitrary method) Data showing the results).

Further, the information processing apparatus according to the present embodiment, for example, if the language of the character string indicated by the speech text information corresponding to the utterance content is different from the set language, the summarized utterance content is You may translate into the set language.

The information processing apparatus according to the present embodiment translates the content of the summarized utterance into another language by processing of an arbitrary algorithm that can be translated into another language.

FIG. 4 shows an example of the result of the translation processing according to this embodiment. FIG. 4A shows the summary result shown in FIG. 3C as an example of the content of the summarized speech before being translated. 4B shows an example of the content of the summary result shown in FIG. 3C translated into another language by the translation process. The summary result shown in FIG. 3C is translated into Japanese. An example is shown. Hereinafter, the translation result obtained by translating the divided text such as the summary result shown in FIG. 3C may be referred to as “divided translation text”.

The content of the utterance summarized as shown in B of FIG. 4 is translated into Japanese, which is the native language of the communication partner U2, so that the communication partner U2 can understand the content that the user U1 is visiting. The possibility of being able to be made can be further increased as compared with the case where the content of the summarized utterance is not translated.

(D) An example of the notification control process according to the second information processing method The information processing apparatus according to the present embodiment notifies the content of the utterance indicated by the voice information summarized by the summarization process shown in (b) above. . Further, when the content of the summarized utterance is translated into another language by further performing the translation processing shown in (c) above, the information processing apparatus according to the present embodiment notifies the translation result. .

As described above, the information processing apparatus according to the present embodiment, for example, summarizes the content of an utterance (or a translation result) by one or both of a notification by a visual method and a notification by an auditory method. , Let the notification content.

FIG. 5 shows an example of the result of the notification control process according to the present embodiment. In FIG. 5, “an example in which the translation result is audibly notified by outputting a voice indicating the translation result from the voice output device connected to the eyewear type device worn by the user U1” Is shown. FIG. 5 shows an example in which the translation result shown in B of FIG. 4 is notified.

FIG. 5 shows an example in which the sound pressure at the location corresponding to the speech location where the sound pressure is strong (the “why” portion shown in FIG. 5) is made stronger than the other locations based on the voice information.

Also, FIG. 5 shows an example in which when a voice indicating the translation result is output, the division of the divided text is notified by inserting a sound feedback as indicated by a symbol “S” in FIG. Yes.

Note that the example of notification realized by the notification control process according to the second information processing method is not limited to the example shown in FIG. Another example of the notification realized by the notification control process according to the second information processing method will be described later.

For example, as shown in FIG. 5, the content (translation result) of the summarized utterance translated into Japanese, which is the native language of the communication partner U2, is output as a notification content from the voice output device by voice, thereby allowing the user to It becomes easier for the communication partner U2 to understand the content that U1 is visiting.

As a use case to which the information processing method according to the present embodiment is applied, there is a use case of “conversation support” (including a case where translation is performed) as described above. As described above, it goes without saying that the use case to which the information processing method according to the present embodiment is applied is not limited to the “conversation support” as described above.

[3] Processing related to the information processing method according to the present embodiment Next, processing related to the information processing method according to the present embodiment will be described more specifically. Below, the summary process which concerns on a 1st information processing method, the translation process which concerns on this embodiment, and the notification control process which concerns on a 2nd information processing method are demonstrated.

[3-1] Summarization Processing According to First Information Processing Method The information processing apparatus according to the present embodiment summarizes the utterance content indicated by the voice information based on the user's utterance based on the information indicating the weight related to the summary. .

As described above, the weight regarding the summary is set based on, for example, one or more of the voice information, the user state, the application execution state, and the user operation. Further, as described above, the information processing apparatus according to the present embodiment summarizes the content of an utterance by an objective function using a weight related to a set summary as shown in Equation 1 above, for example.

Also, the information processing apparatus according to the present embodiment can perform one or more of the following processes (1) to (3), for example, as the summary process.

(1) First example of summary processing: start timing of summary processing The information processing apparatus according to the present embodiment performs summary processing when it is determined that a predetermined start condition that has been set is satisfied.

Examples of the start conditions for the summary processing according to the present embodiment include the following examples.
・ Conditions related to the no-speech period during which no utterance continues ・ Conditions related to the state of speech recognition for acquiring the utterance content from speech information ・ Conditions related to the utterance content ・ Elapsed time since the speech information was obtained Conditions

FIGS. 9A to 9C are explanatory diagrams for explaining an example of the summary processing according to the first information processing method, and show an outline of the start timing of the summary processing. Hereinafter, an example of processing in each start condition will be described with reference to FIGS. 9A to 9C as appropriate.

(1-1) First example of start condition: an example in which the start condition is a condition related to a non-speech period Examples of the condition related to a non-speech period include a condition related to the length of a non-speech period. When the predetermined start condition is a condition related to a non-speech period, the information processing apparatus according to the present embodiment is configured so that the non-speech period is set or the non-speech period is set. When the predetermined period is exceeded, it is determined that the start condition is satisfied.

Here, the period according to the first example of the start condition may be a fixed period that is set in advance, or may be a variable period that can be changed based on a user operation or the like.

Referring to A of FIG. 9A, the “silent period” shown in A of FIG. 9A corresponds to the silent period.

The information processing apparatus according to the present embodiment detects, for example, a voice section in which voice is present based on voice information. Then, the information processing apparatus according to the present embodiment detects the silent period exceeding the set time after the voice section is detected, or detects the silent period longer than the set time. If this is done, the summarization process is started as a summarization process start trigger (hereinafter referred to as “summary trigger”).

(1-2) Second example of start condition: Example in which start condition is first condition related to voice recognition state As a first condition related to voice recognition state, detection of a voice recognition stop request is detected. The conditions concerning are mentioned. When the predetermined start condition is the first condition related to the voice recognition state, the information processing apparatus according to the present embodiment determines that the start condition is satisfied based on the detection of the voice recognition stop request. . The information processing apparatus according to the present embodiment determines that the start condition is satisfied, for example, when a voice recognition stop request is detected.

Referring to B of FIG. 9A, the information processing apparatus according to the present embodiment, for example, after the voice recognition is started based on the “speech recognition start operation” illustrated in B of FIG. 9A, “ When a speech recognition stop request including a speech recognition stop command based on the “speech recognition stop operation” is detected, the summary processing is started as a summary trigger. Here, examples of the voice recognition start operation and the voice recognition stop operation include an operation on an arbitrary UI (User Interface) related to voice recognition.

Note that the speech recognition stop request according to the present embodiment is not limited to being obtained based on the speech recognition stop operation. For example, a speech recognition stop request is generated by a device that performs speech recognition processing when an error occurs during speech recognition processing or when an interrupt processing is entered during speech recognition processing. May be.

(1-3) Third example of start condition: an example in which the start condition is a second condition related to the state of speech recognition The second condition related to the state of speech recognition includes a condition related to completion of speech recognition Is mentioned. When the predetermined start condition is the second condition related to the voice recognition state, the information processing apparatus according to the present embodiment determines that the start condition is satisfied based on the completion of the voice recognition. The information processing apparatus according to the present embodiment determines that the start condition is satisfied, for example, when the completion of voice recognition is detected.

Referring to A of FIG. 9B, the information processing apparatus according to the present embodiment, for example, displays a summary trigger when the result of the speech recognition process is obtained, as indicated by “acquisition of speech recognition result” in A of FIG. As a result, the summarization process is started.

(1-4) Fourth example of start condition: an example in which the start condition is the first condition related to the content of the utterance The first condition related to the content of the utterance is based on the content of the utterance indicated by the voice information. Examples include conditions relating to detection of a predetermined word. When the predetermined start condition is the first condition regarding the content of the utterance, the information processing apparatus according to the present embodiment starts the condition based on the detection of the predetermined word from the content of the utterance indicated by the voice information. Is determined to be satisfied. The information processing apparatus according to the present embodiment determines that the start condition is satisfied, for example, when a predetermined word is detected from the utterance content indicated by the audio information.

Examples of the predetermined word relating to the first condition regarding the content of the utterance include a word called a filler word. The predetermined words related to the first condition relating to the content of the utterance may be preset fixed words that cannot be added, deleted, or changed, or may be added, deleted, or changed based on a user operation or the like. May be possible.

Referring to B of FIG. 9B, “Et” shown in B of FIG. 9B corresponds to an example of filler word (an example of a predetermined word).

The information processing apparatus according to the present embodiment starts the summarization process using, for example, a summary trigger as a case where a filler word is detected from a character string indicated by voice text information obtained based on voice information.

(1-5) Fifth example of start condition: an example in which the start condition is the second condition related to the content of the utterance The second condition related to the content of the utterance is based on the utterance content indicated by the voice information. A condition related to the detection of stagnation is given. When the predetermined start condition is the second condition regarding the content of the utterance, the information processing apparatus according to the present embodiment determines that the start condition is satisfied based on the detection of stagnation based on the voice information. The information processing apparatus according to the present embodiment determines that the start condition is satisfied, for example, when stagnation is detected based on audio information.

The information processing apparatus according to the present embodiment, for example, from “a method of detecting voiced pause (including syllable extension) from voice information” or “a character string indicated by voice text information obtained based on voice information, Say stagnation based on speech information by any method that can detect stagnation based on speech information or estimate speech based on speech information, such as "Method of detecting words associated with sloppyness" Is detected.

Referring to A of FIG. 9C, the information processing apparatus according to the present embodiment starts the summarization process using, for example, a summarization trigger when it is estimated that there is stagnation.

(1-6) Sixth example of start condition: an example in which the start condition is a condition related to the elapsed time since the voice information was obtained. The condition related to the elapsed time after the voice information was obtained is the elapsed time. A condition related to the length of time is given. When the predetermined start condition is a condition related to the elapsed time since the voice information is obtained, the information processing apparatus according to the present embodiment is configured to operate when the elapsed time exceeds a predetermined period, or When the time is equal to or longer than a predetermined period, it is determined that the start condition is satisfied.

Here, the period according to the sixth example of the start condition may be a fixed period set in advance or a variable period that can be changed based on a user operation or the like.

Referring to B of FIG. 9C, the information processing apparatus according to the present embodiment performs, for example, a summarization process when a predetermined time has elapsed since it was detected that audio information was obtained as a summarization trigger. To start.

(1-7) Seventh Example of Start Condition The start conditions are from the start condition according to the first example shown in (1-1) to the start condition according to the sixth example shown in (1-6) above. The condition which combined 2 or more of them may be sufficient. For example, the information processing apparatus according to the present embodiment starts the summarization process when any one of the combined start conditions is satisfied as a summarization trigger.

(2) Second Example of Summarization Processing: Exception Processing without Performing Summarization Processing The information processing apparatus according to the present embodiment sets a summary processing exclusion condition (hereinafter referred to as “summary exclusion condition”). If it is determined that the condition is satisfied, the summarization process is not performed.

As the summary exclusion condition according to this embodiment, for example, a condition related to gesture detection can be given. The information processing apparatus according to the present embodiment determines that the summary exclusion condition is satisfied when a predetermined gesture that has been set is detected.

The predetermined gesture related to the summary exclusion condition may be a fixed gesture set in advance, or may be added, deleted, or changed based on a user operation or the like. For example, the information processing apparatus according to the present embodiment performs image processing on a captured image obtained by imaging with an imaging device, estimates a motion based on a detection result of a motion sensor such as an acceleration sensor or an angular velocity sensor, and the like. Then, it is determined whether or not a predetermined gesture related to the summary exclusion condition has been performed.

Note that the summary exclusion condition according to the present embodiment is not limited to the above-described conditions related to gesture detection.

For example, the summary exclusion condition according to the present embodiment is “the operation for invalidating the function for performing the summary processing, such as pressing a button for invalidating the function for performing the summary processing,” or “ It may be an arbitrary condition set as the summary exclusion condition, such as “the processing load of the information processing apparatus according to the present embodiment is larger than a set threshold”.

(3) Third Example of Summarization Processing: Processing for Dynamically Changing Summarization Level The information processing apparatus according to the present embodiment includes an utterance period identified based on speech information and the number of characters identified based on speech information. Based on one or both of the above, the utterance content summary level (or the utterance content summary level, the same shall apply hereinafter) is changed. In other words, the information processing apparatus according to the present embodiment changes the level of the summary of the utterance content based on at least one of the utterance period specified based on the voice information and the number of characters specified based on the voice information. .

The information processing apparatus according to the present embodiment changes the level of utterance content summarization by, for example, limiting the number of characters indicated by the summarized utterance content. The information processing apparatus according to the present embodiment limits the number of characters indicated by the summarized utterance content, for example, by preventing the number of characters indicated by the summarized utterance content from exceeding the set upper limit value. . By limiting the number of characters indicated by the summarized utterance content, it is possible to automatically reduce the number of characters indicated by the summarized utterance content, that is, the summary amount.

Here, the utterance period is specified, for example, by detecting a voice section in which voice is present based on voice information. Further, the number of characters corresponding to the utterance is specified by counting the number of characters in the character string indicated by the speech text information based on the speech information.

When changing the summary level of the utterance content based on the utterance period, the information processing apparatus according to the present embodiment, for example, when the utterance period exceeds a predetermined period or the utterance period is set When the predetermined period is exceeded, the summary level of the content of the utterance is changed. Here, the above-mentioned period when the level of the summary of the utterance content is changed based on the utterance period may be a fixed period set in advance, or a variable that can be changed based on a user operation or the like. It may be a period.

In addition, when changing the summary level of the content of the utterance based on the number of characters specified based on the voice information, the information processing apparatus according to the present embodiment, for example, when the number of characters is larger than a set threshold, Or, when the number of characters exceeds a set threshold value, the level of utterance content summarization is changed. Here, the threshold in the case of changing the summary level of the utterance content based on the number of characters specified based on the voice information may be a preset fixed threshold, or may be used by a user operation or the like. It may be a variable threshold that can be changed based on this.

[3-2] Translation processing according to this embodiment As shown in (c) above, the information processing apparatus according to this embodiment uses the contents of the utterances summarized by the summarization processing according to the first information processing method. It is possible to further perform a translation process for translating into the above languages. As described above, the information processing apparatus according to the present embodiment translates the first language corresponding to the utterance into a second language different from the first language.

In the translation process, the reliability of the translation result may be set for each translation unit.

Translation unit is a unit that translates in translation processing. As a translation unit, for example, a fixed unit that is set, such as for each word, for each one or two or more phrases, can be cited. Further, the translation unit may be dynamically set according to, for example, a language (first language) corresponding to the utterance. The translation unit may be changeable based on, for example, a user setting operation.

The reliability of the translation result is, for example, an index indicating the certainty of the translation result. For example, 0 [%] (indicating that the reliability is the lowest) to 100 [%] (indicating that the reliability is the highest) (Shown). The reliability of the translation result is obtained by using an arbitrary machine learning result such as a machine learning result using a feedback result with respect to the translation result. Note that the reliability of the translation result is not limited to being obtained using machine learning, but may be obtained by any method capable of obtaining the certainty of the translation result.

In addition, the information processing apparatus according to the present embodiment can perform, for example, one or both of the following (i) and (ii) as the translation processing.

(I) First example of translation processing: exception processing without translation processing The information processing apparatus according to the present embodiment performs translation processing when it is determined that a set translation processing exclusion condition is satisfied. Absent.

Exceptional conditions for translation processing according to the present embodiment include, for example, conditions relating to gesture detection. The information processing apparatus according to the present embodiment determines that the translation process is satisfied when a predetermined gesture that has been set is detected.

The predetermined gesture related to the translation processing may be a fixed gesture set in advance, or may be added, deleted, or changed based on a user operation or the like. Examples of the fixed gesture set in advance include gestures and hand gestures related to non-verbal communication such as hand signs. For example, the information processing apparatus according to the present embodiment performs image processing on a captured image obtained by imaging with an imaging device, estimates a motion based on a detection result of a motion sensor such as an acceleration sensor or an angular velocity sensor, and the like. Then, it is determined whether or not a predetermined gesture related to the translation process has been performed.

It should be noted that the exclusion conditions for the translation processing according to the present embodiment are not limited to the conditions relating to gesture detection as described above.

For example, the exclusion condition for the translation process according to the present embodiment is that “an operation for invalidating the function for performing the translation process, such as pressing a button for invalidating the function for performing the translation process, is detected” or An arbitrary condition set as an exclusion condition for translation processing, such as “the processing load of the information processing apparatus according to the present embodiment has become larger than a set threshold”, may be used. Further, the exclusion conditions for the translation processing according to the present embodiment may be the same conditions as the summary exclusion conditions according to the above-described embodiment, or may be different conditions.

(Ii) Second Example of Translation Processing: Processing in Retranslation The information processing apparatus according to the present embodiment can also retranslate content translated into another language into a language before translation.

The information processing apparatus according to the present embodiment, when an operation for performing re-translation processing is detected, such as when a button for performing re-translation is pressed, for example, , Retranslate to the language before translation.

Note that the retranslation trigger is not limited to the detection of the operation for performing the retranslation processing as described above. For example, the information processing apparatus according to the present embodiment can automatically perform retranslation based on the reliability of the translation result set for each translation unit. For example, the information processing apparatus according to the present embodiment performs retranslation when the reliability of the translation result set for each translation unit is less than or equal to the set threshold value or less than the set threshold value. Re-translate as a trigger.

In addition, when the content translated into another language is re-translated into the language before translation, the information processing apparatus according to the present embodiment may perform a summary process using the result of the re-translation.

As an example, the information processing apparatus according to the present embodiment, for example, in the case where there are words included in the content after retranslation in the content of the utterance indicated by the voice information acquired after retranslation Include the words contained in the re-translated content in the summarized utterance content. By performing the summarization process using the result of retranslation as described above, for example, “When the same words as before retranslation appear in the content uttered by the user, the summary corresponding to this utterance Is adjusted so that the same words are not deleted.

[3-3] Notification Control Process According to Second Information Processing Method The information processing apparatus according to the present embodiment is configured to display the utterance content indicated by the voice information summarized by the summary process according to the first information processing method. Notify me.

As described above, when the content of the summarized utterance is translated into another language by the translation processing according to the present embodiment, the information processing apparatus according to the present embodiment notifies the translation result.

Further, as described above, the information processing apparatus according to the present embodiment notifies the notification content by one or both of notification by a visual method and notification by an auditory method, for example.

FIG. 10 is an explanatory diagram showing an example of notification by a visual method realized by the notification control process according to the second information processing method. FIG. 10 shows an example when the information processing apparatus according to the present embodiment displays the translation result on the display screen of the smartphone.

In addition, the information processing apparatus according to the present embodiment can perform one or more of the following processes (I) to (VII) as the notification control process, for example. Hereinafter, a case where the information processing apparatus according to the present embodiment notifies the translation result will be described as an example. Note that the information processing apparatus according to the present embodiment can also notify the content of the summarized utterance before translation in the same manner as when the translation result is notified.

FIG. 11 to FIG. 21 are explanatory diagrams for explaining an example of the notification control processing according to the second information processing method. Hereinafter, an example of the notification control process according to the second information processing method will be described with reference to FIGS. 11 to 21 as appropriate.

(I) First example of notification control processing: Notification in word order of translated language The information processing apparatus according to the present embodiment notifies the translation result in a word order corresponding to another translated language.

For example, when the content of the utterance is summarized in the divided text as shown in C of FIG. 3 in the summarization process and the other language is English, the information processing apparatus according to the present embodiment Then, notify the translation result.
・ Noun ・ Verb ・ Adjective ・ Adverb ・ Other

Further, for example, when the content of the utterance is summarized into the divided text as shown in C of FIG. 3 in the summary process, when the other language is Japanese, the information processing apparatus according to the present embodiment is The translation results are notified in the following order.
・ Verbs ・ Nouns ・ Adjectives ・ Adverbs ・ Others

As described above, by causing the translation results to be notified in the word order corresponding to the other translated languages, for example, in the word order of the translation results shown in B of FIG. 4 and the auditory notification as shown in FIG. It is possible to change the word order.

Here, the word order corresponding to the other translated languages may be a fixed word order set in advance, or may be changeable based on a user operation or the like.

(II) Second Example of Notification Control Processing: Notification Control Processing Based on Reliability for Each Translation Unit As described above, in translation processing, the reliability of translation results can be set for each translation unit. When the reliability of the translation result is set for each translation unit in the translation process, the information processing apparatus according to the present embodiment notifies the translation result based on the reliability for each translation unit in the summarized utterance content. .

The information processing apparatus according to the present embodiment notifies the translation result based on the reliability for each translation unit by performing one or both of the following processes (II-1) and (II-2), for example. .

(II-1) First Example of Notification Control Processing Based on Reliability for Each Translation Unit The information processing apparatus according to the present embodiment gives priority to notification of translation results with high reliability.

For example, when displaying the translation result visually on the display screen of the display device, the information processing apparatus according to the present embodiment gives a priority notification of the translation result with high reliability depending on how to display the translation result. Realize. In addition, when the translation result is audibly notified by voice from the voice output device, the information processing apparatus according to the present embodiment realizes a high-priority translation result notification according to the order of notification, for example. May be.

In the following, the notification realized by the notification control process based on the reliability for each translation unit according to the first example, taking the case where the translation result is displayed visually on the display screen of the display device as an example. An example will be described.

FIG. 11 shows a first example in the case where the translation result is displayed on the display screen of the display device, and shows an example in which the translation result with high reliability is notified preferentially. In the example shown in FIG. 11, “Recommendation”, “Sightseeing”, “Directions”, “Tell me”, and “Asakusa” correspond to the translation results for each translation unit. FIG. 11 shows an example in which lower reliability is set in the order of “recommended”, “sightseeing”, “direction”, “tell me”, and “Asakusa”.

For example, as illustrated in FIG. 11A, the information processing apparatus according to the present embodiment displays the translation results for each translation unit so that the translation results for each translation unit are hierarchically displayed in an order of high reliability. And display it on the display screen.

Here, hierarchical display is realized by threshold processing using, for example, reliability for each translation unit and one or more thresholds related to determination of the hierarchy to be displayed. Here, the threshold for hierarchical display may be a fixed value set in advance, or may be a variable value that can be changed based on a user operation or the like.

As a result of the threshold processing, when displaying the translation results for each of a plurality of translation units on the same hierarchy, the information processing apparatus according to the present embodiment, for example, “from left to right in the display screen area corresponding to the hierarchy” The translation results for each of the plurality of translation units are displayed in a predetermined order such as “Arrange in order of higher reliability”.

In addition, when there are a plurality of translation results whose reliability is greater than the predetermined threshold or a plurality of translation results whose reliability is equal to or higher than the predetermined threshold as a result of the threshold processing, the information processing apparatus according to the present embodiment For example, as shown in FIG. 11B, a plurality of existing translation results may be displayed together in a predetermined area of the display screen. Here, examples of the predetermined threshold include one or more thresholds among one or more thresholds used for threshold processing. Further, examples of the predetermined area include “display screen area corresponding to a hierarchy associated with the threshold processing based on the predetermined threshold”.

For example, by performing the display as shown in FIG. 11, the translation result for each translation unit for which “high reliability (corresponding to a score) in translation processing” is set is displayed at the top and the reliability is high. When a predetermined threshold value is exceeded, “translation results for each translation unit are displayed together” is realized. Needless to say, the display example in the case where the translation result with high reliability is preferentially notified is not limited to the example shown in FIG.

(II-2) Second Example of Notification Control Processing Based on Reliability for Each Translation Unit The information processing apparatus according to the present embodiment notifies the translation result so as to be emphasized according to the reliability.

For example, when the translation result is displayed visually on the display screen of the display device, the information processing apparatus according to the present embodiment realizes the notice emphasized according to the reliability depending on the display method. In addition, when the translation result is audibly notified by voice from the voice output device, the information processing apparatus according to the present embodiment changes the reliability by changing the sound pressure, the volume, and the like of the voice based on the reliability, for example. A notice that is emphasized according to the above may be realized.

In the following, the notification realized by the notification control process based on the reliability for each translation unit according to the second example will be described by taking as an example the case where the translation result is displayed visually on the display screen of the display device. An example will be described.

The information processing apparatus according to the present embodiment emphasizes and displays the translation result according to the reliability by, for example, “displaying each translation result for each translation unit in a size corresponding to the reliability”.

FIG. 12 shows a second example when the translation result is displayed on the display screen of the display device, and shows a first example when the translation result is displayed in an emphasized manner according to the reliability. . In the example shown in FIG. 12, “Recommendation”, “Sightseeing”, “Directions”, “Tell me”, and “Asakusa” correspond to the translation results for each translation unit. FIG. 12 shows an example in which lower reliability is set in the order of “recommended”, “sightseeing”, “direction”, “tell me”, and “Asakusa”.

FIG. 12 shows the information processing apparatus according to this embodiment in addition to the notification control process based on the reliability for each translation unit according to the first example, in addition to the translation results for each translation unit. An example in which the size is displayed in accordance with is shown. Note that, in the case of performing the notification control process based on the reliability for each translation unit according to the second example, the information processing apparatus according to the present embodiment is reliable as in the hierarchical display shown in FIG. Needless to say, it is not necessary to preferentially notify the translation result having a high degree.

The information processing apparatus according to the present embodiment displays each translation result for each translation unit in a size corresponding to the reliability, for example, as shown in FIG. The information processing apparatus according to the present embodiment refers to, for example, “a table (or database) in which reliability is associated with a display size when displaying a translation result for each translation unit on the display screen”. The translation result for each translation unit is displayed in a size corresponding to the reliability.

For example, by performing the display as shown in FIG. 12, the translation result for each translation unit having a high reliability (corresponding to a score) in the translation process is displayed at the top and displayed at the top. As the result of translation for each translation unit is changed, the size is changed prominently ”. Needless to say, the display example when displaying each translation result for each translation unit in a size corresponding to the reliability is not limited to the example shown in FIG.

In addition, the information processing apparatus according to the present embodiment, for example, provides reliability by “displaying each translation result for each translation unit so that a translation result with high reliability is displayed in the foreground”. Depending on the case, the translation result may be highlighted and displayed.

FIG. 13 shows a third example when the translation result is displayed on the display screen of the display device, and shows a second example when the translation result is displayed with emphasis according to the reliability. . In the example shown in FIG. 13, “Recommendation”, “Sightseeing”, “Direction”, “Tell me”, “Asakusa”, etc. correspond to the translation results for each translation unit. FIG. 13 shows an example in which lower reliability is set in the order of “recommendation”, “sightseeing”, “direction”, “tell me”, “Asakusa”,.

Further, FIG. 13 shows that the information processing apparatus according to the present embodiment displays a translation result with higher reliability on the display screen in addition to the notification control process based on the reliability for each translation unit according to the first example. The example displayed on the near side is shown. As described above, when the notification control process based on the reliability for each translation unit according to the second example is performed, the information processing apparatus according to the present embodiment displays a hierarchical display as illustrated in FIG. Needless to say, it is not necessary to preferentially notify a translation result with a high degree of reliability.

For example, as illustrated in FIG. 13A, the information processing apparatus according to the present embodiment displays a translation result with high reliability on the display screen. The information processing apparatus according to the present embodiment is, for example, “a table (or database) in which the reliability and the coordinate value in the depth direction when displaying the translation result for each translation unit on the display screen are associated with each other”. By referring to, each translation result for each translation unit is displayed so that the translation result with high reliability is displayed on the front side of the display screen.

For example, when the display as shown in FIG. 13 is performed, the translation result for each translation unit having a high reliability (corresponding to a score) in the translation process is displayed on the front in the depth direction on the display screen. As a result, it is realized that the result of translation for each translation unit for which high reliability is set is more conspicuous. In addition, the example of a display in the case of displaying each translation result for every translation unit so that a translation result with high reliability is displayed in the foreground on the display screen is not limited to the example shown in FIG. Needless to say.

Further, the information processing apparatus according to the present embodiment, for example, “displays each translation result for each translation unit in one or both of the color according to the reliability and the transparency according to the reliability”, The translation result may be highlighted and displayed according to the reliability.

FIG. 14 shows a fourth example when the translation result is displayed on the display screen of the display device, and shows a third example when the translation result is displayed in an emphasized manner according to the reliability. . In the example illustrated in FIG. 14, “recommendation”, “tourism”, “direction”, “tell me”, and “Asakusa” correspond to the translation results for each translation unit. FIG. 14 shows an example in which lower reliability is set in the order of “recommendation”, “tourism”, “direction”, “tell me”, and “Asakusa”.

Further, FIG. 14 shows that the information processing apparatus according to the present embodiment further adds the translation result for each translation unit to the reliability in addition to the notification control process based on the reliability for each translation unit according to the first example. An example is shown in which one or both of the color according to the degree and the transparency according to the reliability are displayed. As described above, when the notification control process based on the reliability for each translation unit according to the second example is performed, the information processing apparatus according to the present embodiment displays a hierarchical display as illustrated in FIG. Needless to say, it is not necessary to preferentially notify a translation result with a high degree of reliability.

For example, as illustrated in FIG. 14A, the information processing apparatus according to the present embodiment displays each translation result for each translation unit in a color corresponding to the reliability. In addition, the information processing apparatus according to the present embodiment may display each translation result for each translation unit with a transparency according to the reliability. Furthermore, the information processing apparatus according to the present embodiment can display, for example, each translation result for each translation unit with a color according to the reliability and a transparency according to the reliability.

The information processing apparatus according to the present embodiment has, for example, “reliability, color when displaying the translation result for each translation unit on the display screen, and transparency when displaying the translation result for each translation unit on the display screen. By referring to the “corresponding table (or database)”, each translation result for each translation unit is displayed in one or both of a color corresponding to the reliability and a transparency corresponding to the reliability. .

For example, when the display as shown in FIG. 14 is performed, “translation results for each translation unit for which high reliability (corresponding to a score) is set in the translation process” is more conspicuous. One or both are emphasized ". In addition, the example of a display in the case of displaying each translation result for every translation unit by one or both of the color according to reliability and the transparency according to reliability is not restricted to the example shown in FIG. Needless to say.

(III) Third example of notification control processing: Notification control processing based on audio information When the notification content is displayed visually on the display screen of the display device, the information processing apparatus according to the present embodiment is Based on the voice information, the display method of the notification content is controlled.

The information processing apparatus according to the present embodiment controls how the notification content is displayed based on the audio information, for example, by “displaying the notification content in a size corresponding to the sound pressure or volume specified from the audio information”. To do. The information processing apparatus according to the present embodiment refers to, for example, “a table (or database) in which sound pressure or volume, display size when displaying divided text, and font size are associated with each other”. The notification content is displayed in a size corresponding to the sound pressure or volume specified from the sound information.

Further, when the content of the summarized utterance is translated into another language by the translation processing according to the present embodiment, the information processing apparatus according to the present embodiment controls how to display the notification content. Similarly to the above, it is possible to control the display method of the translation result based on the voice information.

FIG. 15 shows a fifth example in which the translation result is displayed on the display screen of the display device, and shows an example in which the translation result is displayed with emphasis based on the audio information. In the example shown in FIG. 15, “recommendation”, “tourism”, “direction”, “tell me”, and “Asakusa” correspond to the translation results for each translation unit. FIG. 15 shows an example when the sound pressure or volume is lower in the order of “Tell me”, “Direction”, “Recommendation”, “Sightseeing”, and “Asakusa”, for example.

As shown in FIG. 15A, for example, the information processing apparatus according to the present embodiment converts the translation result for each translation unit (the contents of the translated summary utterance) into a sound pressure or volume specified from the sound information. Display in the appropriate size. The information processing apparatus according to the present embodiment, for example, “a table (or database) in which the sound pressure or volume, the display size when displaying the translation result for each translation unit, and the font size” are associated with each other. The translation result is displayed in a size corresponding to the sound pressure or sound volume specified from the sound information.

For example, when the display as shown in FIG. 15 is performed, “the font and the display size are displayed in a large size so that the one with a high sound pressure (or volume) is more conspicuous” is realized. Needless to say, the display example in the case of controlling the display method based on the audio information is not limited to the example shown in FIG.

(IV) Fourth Example of Notification Control Processing: Notification Control Processing Based on Operation Performed on Display Screen When displaying the notification content on the display screen of the display device for visual notification, this embodiment The information processing apparatus changes the content displayed on the display screen based on an operation performed on the display screen.

Here, the operations performed on the display screen include, for example, operations using operation input devices such as buttons, direction keys, a mouse, and a keyboard, and operations on the display screen (when the display device is a touch panel). Arbitrary operations that can be performed on the screen are listed.

The information processing apparatus according to this embodiment performs, for example, one or both of the following processes (IV-1) and (IV-2) to display the display screen based on an operation performed on the display screen. Change the displayed contents.

(IV-1) First Example of Notification Control Processing Based on Operation Performed on Display Screen The information processing apparatus according to the present embodiment is displayed on the display screen based on the operation performed on the display screen. Change the contents. Examples of changing the content displayed on the display screen according to the present embodiment include one or more of the examples shown below.
-Change the display position of the notification content on the display screen (or change the display position of the translation result on the display screen)
・ Deleting part of the notification content displayed on the display screen (or deleting part of the translation result displayed on the display screen)

The information processing apparatus according to the present embodiment changes the display position of the notification content on the display screen (or the display position of the translation result on the display screen) based on an operation performed on the display screen, for example, The content to be presented to the communication partner can be changed manually. In addition, the information processing apparatus according to the present embodiment may display a part of the notification content displayed on the display screen (or the translation result displayed on the display screen based on an operation performed on the display screen. For example, it is possible to manually delete a translation result in which a mistranslation has occurred.

FIGS. 16A to 16C show examples of display screens in the case where contents displayed on the display screen are changed based on operations performed on the display screen. Here, FIG. 16A shows an example of a display when the translation result for each translation unit by the translation process is re-translated. FIG. 16B shows an example of display in the case where a part of the translation result (translated summarized utterance content) for each translation unit displayed on the display screen is deleted. FIG. 16C shows an example of display when the display position of the translation result (translated summarized utterance content) for each translation unit displayed on the display screen is changed.

For example, a case where the user desires to delete “recommendation” which is a part of the translation result for each translation unit displayed on the display screen is taken as an example. When the user selects “recommendation” as indicated by symbol O in A of FIG. 16B, a window W for selecting whether or not to delete is displayed as shown in A of FIG. 16B. When the user selects “Yes” in the window W, “Recommendation” which is a part of the translation result is deleted as shown in B of FIG. 16B. Needless to say, the example of deleting a part of the translation result for each translation unit displayed on the display screen is not limited to the example shown in FIG. 16B.

Also, for example, a case where the user desires to change the display position of “Recommend” and “Tell me” in the translation results for each translation unit displayed on the display screen will be described as an example. For example, when the user selects “Tell me” as indicated by reference numeral O1 in A of FIG. 16C and then designates the position indicated by reference numeral O2 in B of FIG. 16C by a drag operation, as shown in B of FIG. 16C. In addition, the display positions of “recommend” and “tell me” are switched. Needless to say, the example of changing the display position of the translation result for each translation unit displayed on the display screen is not limited to the example shown in FIG. 16C.

(IV-2) Second Example of Notification Control Processing Based on Operation Performed on Display Screen When displaying summarized utterance content (or translation result) on display screen of display device as notification content It may happen that the content of the summarized utterance (or the translation result) cannot be displayed on one screen. As described above, when the content of the summarized utterance (or the translation result) cannot be displayed on one screen, the information processing apparatus according to the present embodiment selects one part of the notification content. And display it on the display screen.

In addition, when one part of the notification content is displayed on the display screen, the information processing apparatus according to the present embodiment is displayed on the display screen based on an operation performed on the display screen. Change the contents. The information processing apparatus according to the present embodiment changes the content displayed on the display screen, for example, by changing the notification content displayed on the display screen from the one part to the other part.

FIGS. 17 and 18 are examples of display screens in the case of changing the translation result (translated summarized utterance content) for each translation unit based on the operation performed on the display screen. Show. Here, FIG. 17 shows an example of a display screen in which the content displayed on the display screen can be changed by a slider-type UI as shown in FIG. FIG. 18 shows an example of a display screen in which the content displayed on the display screen can be changed by a revolver type UI whose display changes by rotating in the depth direction of the display screen.

For example, in the case where the display shown in FIG. 17 is displayed, a case where the user desires to display the content displayed on the display screen is taken as an example. The user operates the slider-type UI by touching an arbitrary part of the slider shown in FIG. 17A, for example, to display the translation result displayed on the display screen from one part to another part. To change.

Further, for example, in the case where the display shown in FIG. 18 is displayed, a case where the user desires to display the contents displayed on the display screen is taken as an example. The user operates the revolver-type UI, for example, by performing a flick operation as indicated by reference numeral O1 in FIG. 18 to change the translation result displayed on the display screen from one part to another part. Change it.

Needless to say, the example of changing the translation result displayed on the display screen is not limited to the examples shown in FIGS.

(V) Fifth Example of Notification Control Processing: Notification Control Processing Based on Voice Operation The information processing apparatus according to this embodiment audibly transmits a translation result from a voice output device based on voice operation. You may be notified.

FIG. 19 shows an example of the case where the translation result is audibly notified based on the voice operation. FIG. 19 shows an example in which the content to be notified to the communication partner is selected from the translation results for each translation unit by the translation processing based on the operation by voice.

For example, when the translation results for each translation unit by the translation processing are “recommendation”, “tourism”, “direction”, and “tell me”, the information processing apparatus according to the present embodiment is shown in FIG. As shown, the retranslated result is notified by voice as indicated by reference numeral “I1” in A of FIG. At this time, the information processing apparatus according to the present embodiment may insert a sound feedback as indicated by a symbol “S” in FIG. 19A at the division of the divided text as shown in FIG. 19A. .

After the re-translation result is notified by voice, when a voice selection operation as indicated by the symbol “O” is detected in B of FIG. 19, the information processing apparatus according to the present embodiment As indicated by reference numeral “I2”, a voice indicating a translation result corresponding to the selection operation by the voice is output from the voice output device. Here, B in FIG. 19 shows an example of a selection operation by voice for designating a number to be notified to the communication partner. Needless to say, the example of the selection operation by voice according to the present embodiment is not limited to the example described above.

FIG. 20 shows another example in the case where the translation result is audibly notified based on the voice operation. FIG. 20 shows an example in which the content to be notified to the communication partner is excluded from the translation results for each translation unit by the translation processing based on the voice operation.

For example, when the translation results for each translation unit by the translation processing are “recommendation”, “tourism”, “direction”, and “tell me”, the information processing apparatus according to the present embodiment is shown in FIG. As shown, the re-translated result is notified by voice as indicated by reference numeral “I1” in A of FIG. Note that the information processing apparatus according to the present embodiment may insert sound feedback at the division of the divided text, as in A of FIG.

After the re-translation result is notified by voice, when an excluding operation by voice as indicated by the symbol “O” is detected in B of FIG. 20, the information processing apparatus according to the present embodiment As indicated by reference numeral “I2”, a voice indicating a translation result corresponding to the selection operation by the voice is output from the voice output device. Here, B in FIG. 20 shows an example of an excluding operation by voice for designating a number that does not require notification to the communication partner. Needless to say, the example of the audio exclusion operation according to the present embodiment is not limited to the example described above.

Note that it goes without saying that the example of the voice operation and the example of the notification based on the voice operation are not limited to the examples shown in FIGS.

(VI) Sixth Example of Notification Control Processing: Notification Control Processing for Controlling Notification Order Dynamically The information processing apparatus according to the present embodiment can also dynamically control the notification order of notification contents. It is.

The information processing apparatus according to the present embodiment controls the notification order of notification contents based on at least one of information corresponding to the first user and information corresponding to the second user, for example. The information corresponding to the first user includes, for example, at least one of information regarding the first user, information regarding the application, and information regarding the device. The information corresponding to the second user includes at least one of information on the second user, information on the application, and information on the device.

The information related to the first user indicates, for example, one or both of the situation where the first user is placed and the state of the first user. Moreover, the information regarding a 2nd user shows the one or both of the condition where the 2nd user is placed, and the state of a 2nd user, for example. Further, as described above, the information related to the application indicates, for example, the execution state of the application. Further, as described above, the information about the device indicates one or both of the device type and the device state, for example.

A method of estimating a situation in which a user (first user or second user) is placed based on, for example, noise around the user detected from voice information (for example, sound other than voice based on speech) It is estimated by processing according to an arbitrary method capable of estimating the situation where the user is placed, such as a method of estimating the situation where the user is present based on the position indicated by the position information. The process of estimating the situation where the user is placed may be performed by the information processing apparatus according to the present embodiment, or may be performed by an external device of the information processing apparatus according to the present embodiment.

In addition, as described above, the user state is an arbitrary behavior estimation process using one or more of the user's biological information, the detection result of the motion sensor, the captured image captured by the imaging device, and the like. Alternatively, it is estimated by an arbitrary emotion estimation process.

FIG. 21 shows an example of display when the notification order is dynamically controlled. FIG. 21A shows an example of the case where the translation result (translated summarized utterance content) for each translation unit is displayed based on the state of the user. Moreover, B of FIG. 21 has shown an example when the translation result for every translation unit by a translation process is displayed based on the execution state of an application. FIG. 21C shows an example of a case where the translation result for each translation unit by the translation process is displayed based on the situation where the user is placed.

FIG. 21A shows an example of display based on the state of the user when the translation results for each translation unit are “recommended”, “tourist”, “direction”, and “tell me”.

For example, when it is recognized that the user's state is “impaired” based on the biometric information, the detection result of the motion sensor, or the like, the information processing apparatus according to the present embodiment uses a verb as illustrated in FIG. Is displayed preferentially, such as by displaying on the leftmost side of the display screen. The information processing apparatus according to the present embodiment specifies the notification order by referring to, for example, “a table (or database) in which the user status and information indicating the display order are associated with each other”.

FIG. 21B shows an example of display based on the execution state of the application when the translation results for each translation unit are “Hokkaido”, “Origin”, “Delicious”, and “Fish”.

For example, in a device associated with a user such as a smartphone possessed by the user, when the type of application being executed is recognized as “meal browser”, the information processing apparatus according to the present embodiment 21 preferentially displays adjectives by displaying the adjectives on the leftmost side of the display screen as shown in FIG. The information processing apparatus according to the present embodiment specifies the notification order by referring to, for example, “a table (or database) in which an application type and information indicating a display order are associated with each other”.

FIG. 21C shows a display based on the situation where the user is placed when the translation result for each translation unit is “Hurry”, “Shibuya”, “Collecting”, and “No time”. An example is shown.

For example, when noise detected from voice information (for example, sound other than voice based on speech) is larger than a set threshold, the information processing apparatus according to the present embodiment recognizes that the user is in a noisy situation. To do. Then, the information processing apparatus according to the present embodiment preferentially displays the noun (or proper noun) by displaying the noun (or proper noun) on the leftmost side of the display screen as shown in FIG. 21C. . The information processing apparatus according to the present embodiment specifies the notification order by referring to, for example, “a table (or database) in which an environment where a user is placed and information indicating a display order are associated”. .

Note that the example of dynamically controlling the notification order is not limited to the example shown in FIG.

For example, when the notification order is dynamically controlled based on two or more of the situation where the user is placed, the user's state, and the application execution state (the notification order is dynamically changed based on a plurality of pieces of information) As an example of control, the information processing apparatus according to the present embodiment has the priority (or priority) set for each of the situation where the user is placed, the state of the user, and the execution state of the application. Based on this, the notification order is specified. The information processing apparatus according to the present embodiment causes notification contents corresponding to an index having a high priority (or priority) to be preferentially notified.

FIG. 21 shows an example of the notification by the visual method. However, as described above, the information processing apparatus according to the present embodiment can also perform the notification by the auditory method.

As described above, the information processing apparatus according to the present embodiment can also dynamically control the notification order based on each piece of information about the device. As an example of dynamically controlling the notification order based on the information about the device, for example, dynamically controlling the notification order according to the processing load of the processor can be cited.

(VII) Sixth Example of Notification Control Processing: Notification Control Processing for Controlling Notification Content Dynamically The information processing apparatus according to this embodiment can also dynamically control the information content of notification content. It is.

The information processing apparatus according to the present embodiment is, for example, notification content information based on one or more of summary information, information corresponding to the first user, information corresponding to the second user, and voice information. The amount is dynamically controlled. Examples of the dynamic change of the information amount include, for example, the following (VII-1) to (VII-5). Needless to say, examples of dynamically changing the information amount are not limited to the examples shown in the following (VII-1) to (VII-5).

(VII-1) Example of dynamic change of notification content based on summary information The information processing apparatus according to the present embodiment includes, for example, “that” and “it” in the content of the summarized utterance indicated by the summary information. Are included, the instruction word (or the translation result of the instruction word) is not notified.
The information processing apparatus according to the present embodiment, for example, if the content of the summarized utterance indicated by the summary information includes a word corresponding to the greeting (or corresponding to the greeting) (Translation result of words to be) is not notified.

(VII-2) Example of dynamic change of notification content based on information corresponding to first user-In the information processing apparatus according to the present embodiment, for example, when the facial expression of the first user is determined to be laughter The amount of information when notifying the notification content is reduced.
The information processing apparatus according to the present embodiment notifies, for example, notification contents when it is determined that the first user's line of sight is facing upward (an example when it is determined that the first user is close to monologue). I won't let you.
The information processing apparatus according to the present embodiment, for example, displays a notification content when a gesture (for example, a pointing gesture) corresponding to an instruction word such as “that”, “it”, “this” is detected. , Do not let me know.
The information processing apparatus according to the present embodiment, for example, notifies all of the notification contents when it is determined that the first user is in a situation where noise is high.

(VII-3) Example of dynamic change of notification content based on information corresponding to second user-In the information processing apparatus according to the present embodiment, for example, when the facial expression of the second user is determined to be laughter The amount of information when notifying the notification content is reduced.
When the second user is a communication partner, the information processing apparatus according to the present embodiment, for example, determines that the second user may not understand the utterance content (for example, the second user When it is determined that the user's line of sight is not suitable for the first user), the amount of information when the notification content is notified is increased.
When the second user is a communication partner, the information processing apparatus according to the present embodiment determines, for example, that the second user is yawning (for example, determines that the second user is bored) Etc.), the amount of information when the notification content is notified is reduced.
When the second user is a communication partner, the information processing apparatus according to the present embodiment increases the amount of information when notifying the notification content when, for example, it is determined that the second user has nodded or consulted. .
When the second user is a communication partner, the information processing apparatus according to the present embodiment, for example, when it is determined that the size of the pupil of the second user is larger than a predetermined size, or When it is determined that the size is equal to or greater than the predetermined size (an example when it is determined that the user is interested), the amount of information when the notification content is notified is increased.
When the second user is a communication partner, the information processing apparatus according to the present embodiment, for example, determines that the second user may not understand the utterance content (for example, the second user For example, when it is determined that the user's hand is not moving), the amount of information when the notification content is notified is increased.
When the second user is a communication partner, the information processing apparatus according to the present embodiment, for example, when it is determined that the inclination of the body of the second user is tilted forward (determined as interested) An example of when the notification is made) increases the amount of information when the notification content is notified.
The information processing apparatus according to the present embodiment, for example, notifies all of the notification contents when it is determined that the second user is in a situation where noise is high.

(VII-4) Example of dynamic change of notification content based on voice information-For example, the information processing apparatus according to the present embodiment has a volume of utterances detected from voice information larger than a predetermined threshold, or When the volume of the utterance is equal to or higher than the predetermined threshold, the notification content is not notified.
The information processing apparatus according to the present embodiment, for example, if the volume of the utterance detected from the voice information is greater than a predetermined threshold or the volume of the utterance is greater than or equal to the predetermined threshold Notify some or all of

(VII-5) An example of dynamic change of notification content based on a combination of a plurality of pieces of information-The information processing apparatus according to the present embodiment is, for example, when the first user and the second user are different, When it is determined that the user's line of sight matches the line of sight of the second user, the amount of information to be notified is increased (information corresponding to the first user and corresponding to the second user) Example of dynamic change of notification content based on information).

[4] Specific Example of Processing According to Information Processing Method According to This Embodiment Next, a specific example of processing according to the above-described information processing method according to this embodiment will be described. Hereinafter, an example of processing in the use case described with reference to FIGS. 1 to 5 will be shown as a specific example of processing related to the information processing method according to the present embodiment.

22 to 33 are flowcharts showing an example of processing related to the information processing method according to the present embodiment. Hereinafter, an example of processing according to the information processing method according to the present embodiment will be described with reference to FIGS. 22 to 33 as appropriate.

The information processing apparatus according to the present embodiment sets a weight related to summarization (hereinafter, sometimes referred to as “weight for summarization function” or simply “weight”) (S100, presetting). The information processing apparatus according to the present embodiment determines a weight related to the summary by determining a weight related to the summary and holding the weight in a recording medium such as a storage unit (described later). An example of the process in step S100 is the process shown in FIG.

Referring to FIG. 23, the information processing apparatus according to the present embodiment acquires data indicating schedule contents from a schedule application (S200).

The information processing apparatus according to the present embodiment is a table (hereinafter referred to as “behavior information summary weight table”) for identifying the behavior recognized from the data indicating the acquired schedule content and the type of weight related to the summary illustrated in FIG. The type of weight related to the summary is determined (S202).

The information processing apparatus according to the present embodiment may indicate a type of weight related to the summary determined in step S202 and a table for specifying the weight related to the summary illustrated in FIG. 6 (hereinafter referred to as “summary table”). )), The weight for the summary is determined (S204).

The information processing apparatus according to the present embodiment performs, for example, the process illustrated in FIG. 23 as the process of step S100 in FIG. Needless to say, the process of step S100 in FIG. 22 is not limited to the process shown in FIG.

Referring to FIG. 22 again, an example of processing related to the information processing method according to the present embodiment will be described. The information processing apparatus according to the present embodiment validates voice input by, for example, starting an application related to voice input (S102).

The information processing apparatus according to the present embodiment determines whether audio information has been acquired (S104). If it is not determined in step S104 that the voice information has been acquired, the information processing apparatus according to the present embodiment does not proceed with the processes in and after step S106 until it is determined that the voice information has been acquired, for example.

If it is determined in step S104 that voice information has been acquired, the information processing apparatus according to the present embodiment analyzes the voice information (S106). The information processing apparatus according to the present embodiment obtains, for example, sound pressure, pitch, average frequency band, and the like by analyzing audio information. The information processing apparatus according to the present embodiment holds the audio information in a recording medium such as a storage unit (described later) (S108).

The information processing apparatus according to the present embodiment sets a weight related to summarization based on voice information or the like (S110). An example of the process in step S110 is the process shown in FIG.

Referring to FIG. 24, the information processing apparatus according to the present embodiment sets weights related to summarization based on, for example, the average frequency of the voice indicated by the voice information (hereinafter sometimes referred to as “input voice”). (S300). An example of the process in step S300 is the process shown in FIG.

24 shows an example in which the process of step S302 is performed after the process of step S300, the process of step S110 of FIG. 22 is not limited to the process shown in FIG. For example, since the process of step S300 and the process of step S302 are independent processes, the information processing apparatus according to the present embodiment can perform the process of step S304 after the process of step S302. The process of S300 and the process of step S302 can also be performed in parallel.

Referring to FIG. 25, the information processing apparatus according to the present embodiment determines whether or not the average frequency band of voice is 300 [Hz] to 550 [Hz] (S400).

If it is determined in step S400 that the average frequency band of the voice is 300 [Hz] to 550 [Hz], the information processing apparatus according to the present embodiment selects “male” as the type of weight related to the summary. Is determined (S402).

If it is not determined in step S400 that the average frequency band of the voice is 300 [Hz] to 550 [Hz], the information processing apparatus according to the present embodiment has an average frequency band of the voice. , 400 [Hz] to 700 [Hz] is determined (S404).

If it is determined in step S404 that the average frequency band of the voice is 400 [Hz] to 700 [Hz], the information processing apparatus according to the present embodiment selects “female” as the type of weight related to the summary. Is determined (S406).

If it is not determined in step S404 that the average frequency band of the voice is 400 [Hz] to 700 [Hz], the information processing apparatus according to the present embodiment does not determine the weight related to the summary.

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 25 as the process of step S300 of FIG. Needless to say, the process of step S300 in FIG. 24 is not limited to the process shown in FIG.

Referring to FIG. 24 again, an example of the process of step S110 in FIG. 22 will be described. For example, the information processing apparatus according to the present embodiment sets a weight related to the summary based on the sound pressure of the sound indicated by the sound information (S302). An example of the processing in step S302 is the processing shown in FIG.

Referring to FIG. 26, the information processing apparatus according to the present embodiment determines a threshold value related to sound pressure based on the distance between the user of the speaker and the communication partner (S500). An example of the process in step S500 is the process shown in FIG.

Referring to FIG. 27, the information processing apparatus according to the present embodiment acquires the distance D between the current communication partner and the image recognition based on the captured image captured by the imaging device (S600).

The information processing apparatus according to the present embodiment performs, for example, the following mathematical formula 2 (S602).

Then, the information processing apparatus according to the present embodiment, for example, performs the calculation of Equation 3 below, and determines the threshold value related to sound pressure by adjusting the threshold value VPWR_thresh_upper related to sound pressure and the threshold value VPWR_thresh_lowre related to sound pressure. (S604).

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 27 as the process of step S500 of FIG. Needless to say, the process of step S500 in FIG. 26 is not limited to the process shown in FIG.

Referring to FIG. 26 again, an example of the process in step S302 in FIG. 24 will be described. The information processing apparatus according to the present embodiment determines whether or not the sound pressure of the sound indicated by the sound information is greater than or equal to a threshold VPWR_thresh_upper related to the sound pressure (S502).

When it is determined in step S502 that the sound pressure of the sound indicated by the sound information is equal to or higher than the threshold VPWR_thresh_upper related to the sound pressure, the information processing apparatus according to the present embodiment selects “anger” as the weight type related to the summary. “Joy” is determined (S504).

In step S502, when it is not determined that the sound pressure of the sound indicated by the sound information is equal to or higher than the threshold VPWR_thresh_upper related to the sound pressure, the information processing apparatus according to the present embodiment has the sound pressure of the sound indicated by the sound information. Then, it is determined whether or not the threshold value VPWR_thresh_lowre relating to the sound pressure is below (S506).

If it is determined in step S506 that the sound pressure of the sound indicated by the sound information is equal to or lower than the threshold VPWR_thresh_lowre related to the sound pressure, the information processing apparatus according to the present embodiment selects “sadness” as the weight type related to the summary. “Uncomfortable”, “pain”, and “anxiety” are determined (S508).

In Step S506, when it is not determined that the sound pressure of the sound indicated by the sound information is equal to or lower than the threshold VPWR_thresh_lowre related to the sound pressure, the information processing apparatus according to the present embodiment does not determine the weight regarding the summary.

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 26 as the process of step S302 of FIG. Needless to say, the process of step S302 in FIG. 24 is not limited to the process shown in FIG.

Referring to FIG. 24 again, an example of the process of step S110 in FIG. 22 will be described. The information processing apparatus according to the present embodiment, for example, analyzes voice information and holds the number of mora and the location of the accent (S304). Note that the process of step S304 may be performed in the process of step S106 of FIG.

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 24 as the process of step S110 of FIG. Needless to say, the process of step S110 in FIG. 22 is not limited to the process shown in FIG.

Referring to FIG. 22 again, an example of processing related to the information processing method according to the present embodiment will be described. The information processing apparatus according to the present embodiment performs voice recognition on voice information (S112). The voice text information is acquired by performing the process of step S112.

When the process of step S112 is performed, the information processing apparatus according to the present embodiment sets a weight related to the summary based on the speech recognition result and the like (S114). An example of the process in step S114 is the process shown in FIG.

Referring to FIG. 28, the information processing apparatus according to the present embodiment sets a weight for summarization based on the language of the character string indicated by the speech text information (S700). An example of the process in step S700 is the process shown in FIG.

Note that FIG. 28 shows an example in which the processing of steps S704 to S710 is performed after the processing of steps S700 and S702, but the processing of step S114 of FIG. 22 is not limited to the processing shown in FIG. For example, since the processes of steps S700 and S702 and the processes of steps S704 to S710 are independent processes, the information processing apparatus according to the present embodiment performs the processes of steps S700 and S702 after the processes of steps SS704 to S710. Alternatively, the processes in steps S700 and S702 and the processes in steps S704 to S710 can be performed in parallel.

Referring to FIG. 29, the information processing apparatus according to the present embodiment estimates the language of the character string indicated by the voice text information (S800). The information processing apparatus according to the present embodiment estimates a language by a process related to an arbitrary method capable of estimating a language from a character string, such as estimation based on matching with a language dictionary.

When the language is estimated in step S800, the information processing apparatus according to the present embodiment determines whether or not the estimated language is Japanese (S802).

If it is determined in step S802 that the estimated language is Japanese, the information processing apparatus according to the present embodiment determines the weight related to the summary so that the weight of the “Japanese verb” is high. (S804).

In step S802, if it is not determined that the estimated language is Japanese, the information processing apparatus according to the present embodiment determines whether the estimated language is English (S806).

If it is determined in step S806 that the estimated language is English, the information processing apparatus according to the present embodiment determines the weight related to the summary so that the weight of “English nouns and verbs” increases. (S808).

In step S806, if it is not determined that the estimated language is English, the information processing apparatus according to the present embodiment does not determine the weight regarding the summary.

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 29 as the process of step S700 of FIG. Needless to say, the process of step S700 of FIG. 28 is not limited to the process shown in FIG.

Referring to FIG. 28 again, an example of the process in step S114 in FIG. 22 will be described. The information processing apparatus according to the present embodiment, for example, analyzes voice information and holds the number of mora and the location of the accent (S702). Note that the process of step S702 may be performed in the process of step S106 of FIG.

The information processing apparatus according to the present embodiment divides a character string indicated by the speech text information (hereinafter, may be referred to as “speech text result”) into morpheme units by natural language processing, and analyzes the corresponding speech information. The results are linked (S704).

The information processing apparatus according to the present embodiment estimates an emotion based on the analysis result of the voice information linked in units of morphemes in step S704 (S706). The information processing apparatus according to the present embodiment can estimate an emotion by using an analysis result of audio information, such as a method of using a table in which an analysis result of audio information and an emotion are associated with each other. Estimate emotions by any method.

In addition, the information processing apparatus according to the present embodiment determines the strength of the weight related to the summary (the strength of the weight related to the emotion) based on the analysis result of the speech information linked in units of morphemes in step S704 ( S708). For example, the information processing apparatus according to the present embodiment determines the strength of the weight related to the summary based on the change rate of the fundamental frequency, the change rate of the sound, and the change rate of the utterance time in the analysis result of the speech information. . The information processing apparatus according to the present embodiment uses, for example, a method of using a table in which an analysis result of speech information is associated with a strength of the summary weight, and the weight related to the summary by using the analysis result of the speech information. The strength of the weight for the summary is determined by any method that can determine the strength of the summary.

The information processing apparatus according to the present embodiment determines a summary weight based on the emotion estimated in step S706 (S710). Further, the information processing apparatus according to the present embodiment may adjust the weight related to the summary determined based on the estimated emotion by the strength of the weight related to the summary determined in step S708.

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 28 as the process of step S114 of FIG. Needless to say, the process of step S114 in FIG. 22 is not limited to the process shown in FIG.

Referring to FIG. 22 again, an example of processing related to the information processing method according to the present embodiment will be described. The information processing apparatus according to the present embodiment performs a summarization process based on the weights related to the summaries determined in steps S100, S110, and S114 (S116).

When the processing in step S116 is completed, the information processing apparatus according to the present embodiment determines whether or not to perform translation processing (S118).

If it is not determined in step S118 that the translation process is to be performed, the information processing apparatus according to the present embodiment notifies the summary result by the notification control process (S120).

If it is determined in step S118 that the translation process is to be performed, the information processing apparatus according to the present embodiment performs the translation process on the summary result and notifies the translation result by the notification control process (S122). An example of the process of step S122 is the process shown in FIG.

Referring to FIG. 30, the information processing apparatus according to the present embodiment performs morphological analysis, for example, by performing natural language processing on the summary result (S900).

The information processing apparatus according to the present embodiment generates a divided text in which the main part of speech (noun, verb, adjective, adverb) and other morphemes are combined until there is no unprocessed summary result (S902).

The information processing apparatus according to the present embodiment determines whether or not the language of the summary result is English (S904).

If it is not determined in step S904 that the language of the summary result is English, the information processing apparatus according to the present embodiment performs the process of step S908 described later.

If it is determined in step S904 that the language of the summary result is English, the information processing apparatus according to the present embodiment sets a word corresponding to 5W1H as a divided text (S906).

When it is not determined in step S904 that the language of the summary result is English, or when the process of step S906 is performed, the information processing apparatus according to the present embodiment performs a translation process on each divided text, and the translation result And the original part-of-speech information before translation are linked and held (S908).

The information processing apparatus according to the present embodiment determines whether or not the language of the divided translation text (an example of the translation result) is English (S910).

When it is determined in step S910 that the language of the divided translation text is English, the information processing apparatus according to the present embodiment determines the notification order in English (S912). An example of the process in step S912 is the process shown in FIG.

Referring to FIG. 31, the information processing apparatus according to the present embodiment determines whether there is a divided translated text to be processed (S1000). Here, the divided translation text to be processed in step S1000 corresponds to an unprocessed translation result among the translation results for each translation unit. The information processing apparatus according to the present embodiment determines, for example, that there is a divided translation text to be processed when there is an unprocessed translation result, and processes the divided translation text when there is no unprocessed translation result. Is determined not to exist.

When it is determined in step S1000 that there is a divided translation text to be processed, the information processing apparatus according to the present embodiment acquires a divided translation text to be processed next (S1002).

The information processing apparatus according to the present embodiment determines whether or not the divided translated text to be processed includes a noun (S1004).

When it is determined in step S1004 that the divided translated text to be processed includes a noun, the information processing apparatus according to the present embodiment sets the priority to the maximum value “5” (S1006). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1000.

If it is not determined in step S1004 that the divided translation text to be processed includes a noun, the information processing apparatus according to the present embodiment determines whether the divided translation text to be processed includes a verb (S1008). .

In step S1008, when it is determined that the divided translated text to be processed includes a verb, the information processing apparatus according to the present embodiment sets the priority to “4” (S1010). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1000.

If it is not determined in step S1008 that the divided translated text to be processed includes a verb, the information processing apparatus according to the present embodiment determines whether the divided translated text to be processed includes an adjective (S1012). .

In step S1012, if it is determined that the divided translated text to be processed includes an adjective, the information processing apparatus according to the present embodiment sets the priority to “3” (S1014). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1000.

In step S1012, if it is not determined that the divided translation text to be processed includes an adjective, the information processing apparatus according to the present embodiment determines whether the divided translation text to be processed includes an adverb (S1016). .

If it is determined in step S1016 that the divided translated text to be processed includes an adverb, the information processing apparatus according to the present embodiment sets the priority to “2” (S1018). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1000.

In step S1016, if it is not determined that the divided translation text to be processed includes an adverb, the information processing apparatus according to the present embodiment sets the priority to the minimum value “1” (S1020). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1000.

If it is not determined in step S1000 that there is a divided translated text to be processed, the information processing apparatus according to the present embodiment sorts the notification order according to the set priority (S1022).

The information processing apparatus according to the present embodiment performs, for example, the process illustrated in FIG. 31 as the process in step S912 in FIG. Needless to say, the process of step S912 in FIG. 30 is not limited to the process shown in FIG.

Referring to FIG. 30 again, an example of the process in step S122 in FIG. 22 will be described. If it is not determined in step S910 that the language of the divided translated text is English, the information processing apparatus according to the present embodiment determines the notification order in Japanese (S914). An example of the process in step S914 is the process shown in FIG.

Referring to FIG. 32, the information processing apparatus according to the present embodiment determines whether or not there is a divided translated text to be processed, similar to step S1100 of FIG. 31 (S1100). Here, the divided translation text to be processed in step S1100 corresponds to an unprocessed translation result among the translation results for each translation unit.

When it is determined in step S1100 that there is a divided translation text to be processed, the information processing apparatus according to the present embodiment acquires a divided translation text to be processed next (S1102).

The information processing apparatus according to the present embodiment determines whether or not the divided translated text to be processed includes a verb (S1104).

When it is determined in step S1104 that the divided translated text to be processed includes a verb, the information processing apparatus according to the present embodiment sets the priority to the maximum value “5” (S1106). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1100.

If it is not determined in step S1104 that the divided translation text to be processed includes a verb, the information processing apparatus according to the present embodiment determines whether or not the divided translation text to be processed includes a noun (S1108). .

If it is determined in step S1108 that the divided translated text to be processed includes a noun, the information processing apparatus according to the present embodiment sets the priority to “4” (S1110). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1100.

If it is not determined in step S1108 that the divided translation text to be processed includes a noun, the information processing apparatus according to the present embodiment determines whether the divided translation text to be processed includes an adjective (S1112). .

If it is determined in step S1112 that the divided translated text to be processed includes an adjective, the information processing apparatus according to the present embodiment sets the priority to “3” (S1114). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1100.

If it is not determined in step S1112 that the divided translation text to be processed includes an adjective, the information processing apparatus according to the present embodiment determines whether the divided translation text to be processed includes an adverb (S1116). .

If it is determined in step S1116 that the divided translated text to be processed includes an adverb, the information processing apparatus according to the present embodiment sets the priority to “2” (S1118). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1100.

If it is not determined in step S1116 that the divided translated text to be processed includes an adverb, the information processing apparatus according to the present embodiment sets the priority to the minimum value “1” (S1120). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1100.

If it is not determined in step S1100 that there is a divided translated text to be processed, the information processing apparatus according to the present embodiment sorts the notification order according to the set priority (S1122).

The information processing apparatus according to the present embodiment performs, for example, the process illustrated in FIG. 32 as the process of step S914 in FIG. Needless to say, the process of step S914 in FIG. 30 is not limited to the process shown in FIG.

Referring to FIG. 30 again, an example of the process in step S122 in FIG. 22 will be described. When the process of step S912 or the process of step S914 is completed, the information processing apparatus according to the present embodiment causes the notification control process to notify the divided translated text for which the notification order is determined (S916). An example of the process in step S916 is the process shown in FIG.

Referring to FIG. 33, the information processing apparatus according to the present embodiment determines whether or not there is a divided translated text to be processed, similar to step S1000 in FIG. 31 (S1200). Here, the divided translation text to be processed in step S1200 corresponds to an unprocessed translation result among the translation results for each translation unit.

When it is determined in step S1200 that there is a divided translation text to be processed, the information processing apparatus according to the present embodiment acquires a divided translation text to be processed next (S1202).

The information processing apparatus according to the present embodiment acquires the sound pressure from the speech information corresponding to the divided translated text to be processed, and increases the sound pressure of the divided translated text to be processed for output (S1204).

The information processing apparatus according to the present embodiment determines whether or not the divided translated text output in step S1204 is the last divided translated text (S1206). The information processing apparatus according to the present embodiment determines, for example, that there is an unprocessed translation result and determines that it is not the last divided translated text, and if there is no unprocessed translation result, the last divided translated text It is determined that

If it is not determined in step S1206 that the text is the last divided translated text, the information processing apparatus according to the present embodiment outputs a “beep” sound as sound feedback for notifying that it will continue thereafter (S1208). ). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1200.

If it is determined in step S1206 that the text is the last divided translated text, the information processing apparatus according to the present embodiment outputs a “beep” sound as sound feedback for notifying the end. (S1210). Then, the information processing apparatus according to the present embodiment repeats the processing from step S1200.

If it is not determined in step S1200 that there is a divided translated text to be processed, the information processing apparatus according to the present embodiment ends the process of FIG.

The information processing apparatus according to the present embodiment performs, for example, the process shown in FIG. 33 as the process of step S916 in FIG. Needless to say, the process of step S916 in FIG. 30 is not limited to the process shown in FIG.

For example, the use cases described with reference to FIGS. 1 to 5 can be realized by performing the processes shown in FIGS. Needless to say, the processing related to the information processing method according to the present embodiment is not limited to the processing shown in FIGS.

[5] An example of an effect produced by using the information processing method according to the present embodiment When the information processing apparatus according to the present embodiment performs processing according to the information processing method according to the present embodiment, for example, the following The effect shown is produced. Needless to say, the effects produced by using the information processing method according to the present embodiment are not limited to the effects described below.
・ Even if the speaker speaks in an uncoordinated manner, only the main points are translated, and it is possible to convey to the receiver what the speaker wants to convey.
・ By translating only the main points, the confirmation time for the recipient can be shortened, and smooth translation communication can be realized.
-In some cases, the sentences to be processed can be extremely reduced, and the accuracy of the translation itself can be improved.
・ Since the content of the utterance is summarized and translated, the recipient does not have to receive unnecessary words, so the recipient is easy to understand. As a result, it is possible to encourage those who are not good at foreign languages to speak across language barriers.

(Information processing apparatus according to this embodiment)
Next, an example of the configuration of the information processing apparatus according to the present embodiment capable of performing the processing according to the information processing method according to the present embodiment described above will be described. Hereinafter, as an example of the configuration of the information processing apparatus according to the present embodiment, one or both of the processing related to the first information processing method described above and the processing related to the second information processing method described above can be performed. An example of an information processing apparatus is shown.

FIG. 34 is a block diagram illustrating an example of the configuration of the information processing apparatus 100 according to the present embodiment. The information processing apparatus 100 includes, for example, a communication unit 102 and a control unit 104.

The information processing apparatus 100 is operated by, for example, a ROM (Read Only Memory. Not shown), a RAM (Random Access Memory. Not shown), a storage unit (not shown), or a user of the information processing apparatus 100. A possible operation unit (not shown), a display unit (not shown) for displaying various screens on the display screen, and the like may be provided. The information processing apparatus 100 connects the above constituent elements by, for example, a bus as a data transmission path. The information processing apparatus 100 is driven by, for example, power supplied from an internal power supply such as a battery provided in the information processing apparatus 100, or power supplied from a connected external power supply.

A ROM (not shown) stores control data such as a program used by the control unit 104 and calculation parameters. A RAM (not shown) temporarily stores a program executed by the control unit 104.

The storage unit (not shown) is a storage unit included in the information processing apparatus 100. For example, the storage unit (not shown) includes various data such as a table for setting weights related to the summary, data related to the information processing method according to the present embodiment, various applications, Memorize data. Here, examples of the storage unit (not shown) include a magnetic recording medium such as a hard disk, and a non-volatile memory such as a flash memory. Further, the storage unit (not shown) may be detachable from the information processing apparatus 100.

As the operation unit (not shown), an operation input device to be described later can be cited. Moreover, as a display part (not shown), the display device mentioned later is mentioned.

[Hardware Configuration Example of Information Processing Apparatus 100]
FIG. 35 is an explanatory diagram illustrating an example of a hardware configuration of the information processing apparatus 100 according to the present embodiment. The information processing apparatus 100 includes, for example, an MPU 150, a ROM 152, a RAM 154, a recording medium 156, an input / output interface 158, an operation input device 160, a display device 162, and a communication interface 164. In addition, the information processing apparatus 100 connects each component with a bus 166 as a data transmission path, for example.

The MPU 150 is composed of, for example, one or two or more processors configured by an arithmetic circuit such as an MPU, various processing circuits, and the like, and functions as the control unit 104 that controls the information processing apparatus 100 as a whole. Further, the MPU 150 plays a role of, for example, the processing unit 110 described later in the information processing apparatus 100. The processing unit 110 may be configured with a dedicated (or general-purpose) circuit (for example, a processor separate from the MPU 150) that can realize the processing of the processing unit 110.

The ROM 152 stores programs used by the MPU 150, control data such as calculation parameters, and the like. The RAM 154 temporarily stores a program executed by the MPU 150, for example.

The recording medium 156 functions as a storage unit (not shown), and stores various data such as data related to the information processing method according to the present embodiment such as a table for setting weights related to summarization and various applications. To do. Here, examples of the recording medium 156 include a magnetic recording medium such as a hard disk and a non-volatile memory such as a flash memory. Further, the recording medium 156 may be detachable from the information processing apparatus 100.

The input / output interface 158 connects, for example, the operation input device 160 and the display device 162. The operation input device 160 functions as an operation unit (not shown), and the display device 162 functions as a display unit (not shown). Here, examples of the input / output interface 158 include a USB (Universal Serial Bus) terminal, a DVI (Digital Visual Interface) terminal, an HDMI (High-Definition Multimedia Interface) (registered trademark) terminal, and various processing circuits. .

The operation input device 160 is provided on the information processing apparatus 100, for example, and is connected to the input / output interface 158 inside the information processing apparatus 100. Examples of the operation input device 160 include a button, a direction key, a rotary selector such as a jog dial, or a combination thereof.

The display device 162 is provided on the information processing apparatus 100, for example, and is connected to the input / output interface 158 inside the information processing apparatus 100. Examples of the display device 162 include a liquid crystal display (Liquid Crystal Display), an organic EL display (Organic Electro-Luminescence Display, or an OLED display (Organic Light Emitting Diode Display)), and the like.

It goes without saying that the input / output interface 158 can be connected to an external device such as an operation input device (for example, a keyboard or a mouse) external to the information processing apparatus 100 or an external display device. The display device 162 may be a device capable of display and user operation, such as a touch panel.

The communication interface 164 is a communication unit included in the information processing apparatus 100, and is a communication unit 102 for performing wireless or wired communication with, for example, an external device or an external device via a network (or directly). Function as. Here, as the communication interface 164, for example, a communication antenna and an RF (Radio Frequency) circuit (wireless communication), an IEEE 802.15.1 port and a transmission / reception circuit (wireless communication), an IEEE 802.11 port and a transmission / reception circuit (wireless communication). ), Or a LAN (Local Area Network) terminal and a transmission / reception circuit (wired communication).

The information processing apparatus 100 performs a process related to the information processing method according to the present embodiment, for example, with the configuration illustrated in FIG. Note that the hardware configuration of the information processing apparatus 100 according to the present embodiment is not limited to the configuration illustrated in FIG.

For example, the information processing apparatus 100 may not include the communication interface 164 when communicating with an external apparatus or the like via a connected external communication device. Further, the communication interface 164 may be configured to be able to communicate with one or more external devices or the like by a plurality of communication methods.

In addition, the information processing apparatus 100 can have a configuration that does not include the recording medium 156, the operation input device 160, and the display device 162, for example.

In addition, the information processing apparatus 100 includes, for example, one or more of various sensors such as a motion sensor and a biological sensor, a voice input device such as a microphone, a voice output device such as a speaker, a vibration device, and an imaging device. Furthermore, you may provide.

Also, for example, part or all of the configuration shown in FIG. 35 (or the configuration according to the modification) may be realized by one or two or more ICs.

Referring to FIG. 34 again, an example of the configuration of the information processing apparatus 100 will be described. The communication unit 102 is a communication unit included in the information processing apparatus 100, and performs wireless or wired communication with, for example, an external apparatus or an external device via a network (or directly). The communication of the communication unit 102 is controlled by the control unit 104, for example.

Here, examples of the communication unit 102 include a communication antenna and an RF circuit, a LAN terminal, and a transmission / reception circuit, but the configuration of the communication unit 102 is not limited to the above. For example, the communication unit 102 can have a configuration corresponding to an arbitrary standard capable of performing communication such as a USB terminal and a transmission / reception circuit, or an arbitrary configuration capable of communicating with an external device via a network. Further, the communication unit 102 may be configured to be able to communicate with one or more external devices or the like by a plurality of communication methods.

The control unit 104 is configured by, for example, an MPU and plays a role of controlling the entire information processing apparatus 100. In addition, the control unit 104 includes, for example, a processing unit 110 and plays a role of leading the processing related to the information processing method according to the present embodiment. For example, the processing unit 110 plays a role of leading one or both of the processing related to the first information processing method and the processing related to the second information processing method.

When performing the process related to the first information processing method described above, the processing unit 110 performs a summarization process for summarizing the content of the utterance indicated by the voice information, based on the acquired information indicating the weight related to the summary. The processing unit 110 performs, for example, the process described in [3-1] as the summary process.

When performing the processing related to the second information processing method described above, the processing unit 110 performs notification control processing for controlling notification of notification contents based on the summary information. The processing unit 110 performs, for example, the process described in [3-3] as the notification control process.

Further, the processing unit 110 may further perform a translation process for translating the content of the utterance summarized by the summarization process into another language. The processing unit 110 performs, for example, the process described in [3-2] as the translation process.

When the content of the utterance summarized by the translation process is translated into another language, the processing unit 110 can notify the translation result by the notification control process.

In addition, the processing unit 110 performs, for example, a process related to speech recognition, a process related to speech analysis, a process related to estimation of a user's state, a process related to estimation of a distance between a user and a communication partner, and the like. Various processes related to the information processing method according to the embodiment can also be performed. Various processes related to the information processing method according to the present embodiment may be performed in an external device of the information processing apparatus 100.

The information processing apparatus 100 has, for example, the configuration shown in FIG. 34 to perform processing related to the information processing method according to this embodiment (for example, “summarization processing related to the first information processing method and notification control related to the second information processing method” One or both of the processing ”,“ one or both of the summary processing according to the first information processing method and the notification control processing according to the second information processing method, and the translation processing ”).

Therefore, when performing the summarization process according to the first information processing method as the process according to the information processing method according to the present embodiment, the information processing apparatus 100 may summarize the content of the utterance with the configuration illustrated in FIG. 34, for example. it can.

In addition, when performing the notification control process according to the second information processing method as the process according to the information processing method according to the present embodiment, the information processing apparatus 100 uses the configuration illustrated in FIG. Can be notified.

Also, for example, with the configuration shown in FIG. 34, the information processing apparatus 100 can achieve the effects that are achieved by performing the processing related to the information processing method according to the present embodiment as described above.

Note that the configuration of the information processing apparatus according to the present embodiment is not limited to the configuration shown in FIG.

For example, the information processing apparatus according to the present embodiment can include the processing unit 110 illustrated in FIG. 34 separately from the control unit 104 (for example, realized by another processing circuit). Further, for example, the summary processing according to the first information processing method, the notification control processing according to the second information processing method, and the translation processing according to the present embodiment may be performed in a distributed manner by a plurality of processing circuits.

For the sake of convenience, the summary processing according to the first information processing method, the notification control processing according to the second information processing method, and the translation processing according to the present embodiment define the processing according to the information processing method according to the present embodiment. It is a thing. Therefore, the configuration for realizing the processing according to the information processing method according to the present embodiment is not limited to the configuration illustrated in FIG. 34, and the configuration according to the method of dividing the processing according to the information processing method according to the present embodiment is taken. It is possible.

Further, for example, when communicating with an external device via an external communication device having the same function and configuration as the communication unit 102, the information processing apparatus according to the present embodiment does not include the communication unit 102. Also good.

As described above, the information processing apparatus has been described as the present embodiment, but the present embodiment is not limited to such a form. The present embodiment is used by being worn on a user's body, such as a “computer such as a personal computer (PC) or a server” or an “eyewear type device, a clock type device, a bracelet type device, etc.” Processing related to the information processing method according to the present embodiment, such as “any wearable device”, “communication device such as a smartphone”, “tablet-type device”, “game machine”, “mobile object such as an automobile”, etc. The present invention can be applied to various devices capable of performing one or both of the processing related to the first information processing method and the processing related to the second information processing method. In addition, the present embodiment can be applied to a processing IC that can be incorporated in the above-described device, for example.

In addition, the information processing apparatus according to the present embodiment may be applied to a processing system that is premised on connection to a network (or communication between apparatuses), such as cloud computing. As an example of a processing system in which processing according to the information processing method according to the present embodiment is performed, for example, “summary processing and translation processing according to the first information processing method are performed by one apparatus configuring the processing system, And a system in which notification control processing according to the second information processing method is performed by another device constituting the system.

(Program according to this embodiment)
[I] Program according to first information processing method (computer program)
A program for causing a computer to function as the information processing apparatus according to the present embodiment that performs processing according to the first information processing method (for example, “summarization processing according to the first information processing method” or “first information A program capable of executing processing related to the first information processing method, such as “summarization processing related to the processing method and translation processing related to the present embodiment”, is executed by a processor or the like in the computer. The contents of can be summarized.

In addition, a program for causing a computer to function as the information processing apparatus according to the present embodiment that performs processing according to the first information processing method is executed by a processor or the like in the computer, whereby the first information processing described above is performed. The effect produced by the process according to the method can be produced.

[II] Program Related to Second Information Processing Method A program for causing a computer to function as the information processing apparatus according to the present embodiment that performs processing related to the second information processing method (for example, “second information processing method” A program capable of executing processing related to the second information processing method such as “notification control processing related to the above”, “translation processing related to the present embodiment, and notification control processing related to the second information processing method”) However, the contents of the summarized utterance can be notified by being executed by a processor or the like in the computer.

In addition, a program for causing a computer to function as the information processing apparatus according to the present embodiment that performs processing according to the second information processing method is executed by a processor or the like in the computer, whereby the second information processing described above is performed. The effect produced by the process according to the method can be produced.

[III] Program related to information processing method according to this embodiment The program related to the information processing method according to this embodiment includes a program related to the first information processing method and a program related to the second information processing method. Both of them may be included.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

For example, in the above, a program for causing a computer to function as the information processing apparatus according to the present embodiment (one or both of the processing related to the first information processing method and the processing related to the second information processing method are executed) However, the present embodiment can also provide a recording medium in which the program is stored.

The above-described configuration shows an example of the present embodiment, and naturally belongs to the technical scope of the present disclosure.

In addition, the effects described in this specification are merely illustrative or illustrative, and are not limited. That is, the technology according to the present disclosure can exhibit other effects that are apparent to those skilled in the art from the description of the present specification in addition to or instead of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
An information processing apparatus (2) including a processing unit that performs a summarization process for summarizing the content of the utterance indicated by the voice information based on the user's utterance based on the acquired information indicating the weight related to the summary.
The information processing apparatus according to (1), wherein the processing unit performs the digest process when it is determined that a predetermined start condition is satisfied.
(3)
The start condition is a condition related to a non-speech period in which a state in which no speech is made continues.
The processing unit determines that the start condition is satisfied when the non-speech period exceeds a predetermined period or when the non-speech period becomes equal to or greater than the predetermined period, (2) Information processing device.
(4)
The start condition is a condition related to a state of speech recognition for acquiring the content of an utterance from the speech information,
The information processing apparatus according to (2) or (3), wherein the processing unit determines that the start condition is satisfied based on detection of the voice recognition stop request.
(5)
The start condition is a condition related to a state of speech recognition for acquiring the content of an utterance from the speech information,
The information processing apparatus according to any one of (2) to (4), wherein the processing unit determines that the start condition is satisfied based on detection of completion of the voice recognition.
(6)
The start condition is a condition related to the content of the utterance,
The processing unit determines that the start condition is satisfied based on detection of a predetermined word from the content of the utterance indicated by the audio information, according to any one of (2) to (5) Information processing device.
(7)
The start condition is a condition related to the content of the utterance,
The information processing apparatus according to any one of (2) to (6), wherein the processing unit determines that the start condition is satisfied based on detection of stagnation based on the audio information.
(8)
The start condition is a condition related to an elapsed time after the voice information is obtained,
The processing unit determines that the start condition is satisfied when the elapsed time exceeds a predetermined period or when the elapsed time is equal to or longer than the predetermined period. (2) to (7) The information processing apparatus according to any one of the above.
(9)
The information processing apparatus according to any one of (1) to (8), wherein the processing unit does not perform the summary processing when it is determined that a predetermined summary exclusion condition is satisfied.
(10)
The summary exclusion condition is a condition related to gesture detection,
The information processing apparatus according to (9), wherein the processing unit determines that the summary exclusion condition is satisfied when a predetermined gesture is detected.
(11)
The processing unit changes a summary level of the content of the utterance based on at least one of an utterance period specified based on the voice information and a number of characters specified based on the voice information. The information processing apparatus according to any one of (10) to (10).
(12)
The information processing apparatus according to (11), wherein the processing unit changes a summary level of the utterance content by limiting a number of characters indicated by the summarized utterance content.
(13)
The processing unit sets a weight for the summary based on at least one of the audio information, information about a user, information about an application, information about an environment, and information about a device, (1) to (12) The information processing apparatus according to any one of the above.
(14)
The information processing apparatus according to (13), wherein the information about the user includes at least one of the user status information and the user operation information.
(15)
The information processing apparatus according to any one of (1) to (14), wherein the processing unit further performs a translation process for translating the content of the speech summarized by the summary process into another language.
(16)
The information processing apparatus according to (15), wherein the processing unit does not perform the translation process when it is determined that a predetermined translation exclusion condition is satisfied.
(17)
The processor is
The content translated into another language by the translation process is re-translated into the language before translation,
If there is a word included in the re-translated content in the utterance content indicated by the speech information acquired after re-translation, the words included in the re-translated content are summarized. The information processing apparatus according to (15) or (16), which is included in the content of the uttered speech.
(18)
The information processing apparatus according to any one of (1) to (17), wherein the processing unit further performs notification control processing for controlling notification of the content of the summarized utterance.
(19)
An information processing method executed by an information processing apparatus, comprising: performing a summarization process for summarizing the content of an utterance indicated by voice information based on a user's utterance based on information indicating a weight related to the acquired summary.
(20)
A program for causing a computer to realize a function of performing a summarization process for summarizing the content of utterances indicated by voice information based on a user's utterances based on information indicating weights relating to acquired summaries.

DESCRIPTION OF SYMBOLS 100 Information processing apparatus 102 Communication part 104 Control part 110 Processing part

Claims

An information processing apparatus comprising a processing unit that performs a summarization process for summarizing the content of an utterance indicated by voice information based on a user's utterance based on information indicating a weight related to the acquired summary.
The information processing apparatus according to claim 1, wherein the processing unit performs the summarization process when it is determined that a predetermined start condition is satisfied.
The start condition is a condition related to a non-speech period in which a state in which no speech is made continues.
The said processing part determines with satisfy | filling the said start conditions, when the said non-utterance period exceeds predetermined period, or when the said non-utterance period becomes more than the said predetermined period. Information processing device.
The start condition is a condition related to a state of speech recognition for acquiring the content of an utterance from the speech information,
The information processing apparatus according to claim 2, wherein the processing unit determines that the start condition is satisfied based on detection of the voice recognition stop request.
The start condition is a condition related to a state of speech recognition for acquiring the content of an utterance from the speech information,
The information processing apparatus according to claim 2, wherein the processing unit determines that the start condition is satisfied based on detection of completion of the voice recognition.
The start condition is a condition related to the content of the utterance,
The information processing apparatus according to claim 2, wherein the processing unit determines that the start condition is satisfied based on detection of a predetermined word from the content of the utterance indicated by the audio information.
The start condition is a condition related to the content of the utterance,
The information processing apparatus according to claim 2, wherein the processing unit determines that the start condition is satisfied based on detection of stagnation based on the audio information.
The start condition is a condition related to an elapsed time after the voice information is obtained,
The information according to claim 2, wherein the processing unit determines that the start condition is satisfied when the elapsed time exceeds a predetermined period or when the elapsed time becomes equal to or longer than the predetermined period. Processing equipment.
The information processing apparatus according to claim 1, wherein the processing unit does not perform the summarization process when it is determined that a predetermined summarization exclusion condition is satisfied.
The summary exclusion condition is a condition related to gesture detection,
The information processing apparatus according to claim 9, wherein the processing unit determines that the summary exclusion condition is satisfied when a predetermined gesture is detected.
The processing unit changes a summary level of the content of the utterance based on at least one of an utterance period specified based on the voice information and a number of characters specified based on the voice information. The information processing apparatus according to 1.
12. The information processing apparatus according to claim 11, wherein the processing unit changes a summary level of the utterance content by limiting a number of characters indicated by the summarized utterance content.
The information according to claim 1, wherein the processing unit sets a weight related to the summary based on at least one of the audio information, information about a user, information about an application, information about an environment, and information about a device. Processing equipment.
14. The information processing apparatus according to claim 13, wherein the information about the user includes at least one of the user status information and the user operation information.
The information processing apparatus according to claim 1, wherein the processing unit further performs a translation process for translating the content of the utterance summarized by the summary process into another language.
The information processing apparatus according to claim 15, wherein the processing unit does not perform the translation processing when it is determined that a predetermined translation exclusion condition is satisfied.
The processor is
The content translated into another language by the translation process is re-translated into the language before translation,
If there is a word included in the re-translated content in the utterance content indicated by the speech information acquired after re-translation, the words included in the re-translated content are summarized. The information processing device according to claim 15, wherein the information processing device is included in the content of the uttered speech.
The information processing apparatus according to claim 1, wherein the processing unit further performs notification control processing for controlling notification of the content of the summarized utterance.
An information processing method executed by the information processing apparatus, including a step of performing a summarization process for summarizing the content of the utterance indicated by the voice information based on the user's utterance based on the information indicating the weight related to the acquired summary.
A program for causing a computer to realize a function of performing a summarization process for summarizing the content of speech indicated by voice information based on a user's speech based on information indicating the weight related to the acquired summary.