JP2022092178A - Information processing device - Google Patents

Abstract

To create minutes based on voice signals.SOLUTION: An information processing device 1 includes a voice recognition unit 13, a conversion unit 14, and a comparison processing unit 15. The voice recognition unit 13 recognizes a voice. The conversion unit 14 converts the voice recognized by the voice recognition unit 13 into character data. The comparison processing unit 15 compares the character data converted by the conversion unit 14 with manuscript data, and if the character data and the manuscript data match, changes the property of the corresponding character in the manuscript data. The voice recognition unit 13 recognizes a specific voice and registers it as a speaker. The conversion unit 14 distinguishes between the voice by the speaker and a voice by a person other than the speaker, and converts each voice into character data. The comparison processing unit 15 changes the character corresponding to the character data in which the voice by the speaker is converted in the manuscript data and the character corresponding to the character data in which the voice by the person other than the speaker is converted to different properties.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device.

Patent Document 1 discloses a system for confirming whether or not all the keywords to be explained by the person in charge can be explained in the negotiation between the customer and the person in charge.

Specifically, the system of Patent Document 1 includes a customer terminal, a person in charge terminal, and a management server. The person in charge terminal picks up the person in charge voice with a microphone and sends it to the management server as a person in charge voice signal. The management server extracts keyword data from the voice signal of the person in charge and compares the extracted keyword data with the preset confirmation keyword data.

Japanese Unexamined Patent Publication No. 2015-99474

In the system of Patent Document 1, it is necessary to set keywords in advance. For example, when a customer asks a question during an explanation by a person in charge, the content of the question is recorded and the answer to the question is appropriate. It is not possible to determine if there is any. Therefore, the system of Patent Document 1 is not suitable for preparing minutes.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an information processing apparatus capable of creating minutes based on an audio signal.

The information processing apparatus according to the present invention includes a voice recognition unit, a conversion unit, and a comparison processing unit. The voice recognition unit recognizes voice. The conversion unit converts the voice recognized by the voice recognition unit into character data. The comparison processing unit compares the character data converted by the conversion unit with the manuscript data, and when the character data and the manuscript data match, the property of the corresponding character in the manuscript data is changed. The voice recognition unit recognizes a specific voice and registers it as a speaker. The conversion unit distinguishes between the voice by the speaker registered by the voice recognition unit and the voice by a person other than the speaker, and converts each of the voices into the character data. The comparison processing unit differs between the character corresponding to the character data in which the voice by the speaker in the manuscript data is converted and the character corresponding to the character data in which the voice by a person other than the speaker is converted. Change to the above property.

According to the present invention, it is possible to create minutes based on an audio signal.

It is a figure which shows the conference system 10 which includes the information processing apparatus 1 which concerns on this embodiment. It is a figure which shows the information processing apparatus 1 which concerns on this embodiment. It is a figure which shows an example of the comparison processing of voice and manuscript data by the information processing apparatus 1 which concerns on this embodiment. It is a figure which shows an example of the property change of the character included in the manuscript data by the information processing apparatus 1 which concerns on this embodiment. It is a figure which shows an example of the re-change of the character property by the information processing apparatus 1 which concerns on this embodiment. It is a flowchart which shows the comparison process which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the figure, the same or corresponding parts are designated by the same reference numerals and the description is not repeated.

First, with reference to FIG. 1, the configuration of the conference system 10 including the information processing apparatus 1 according to the present embodiment will be described. FIG. 1 is a diagram showing a conference system 10 including an information processing device 1 according to the present embodiment. As shown in FIG. 1, the conference system 10 includes an information processing device 1 and a display device 3.

The conference system 10 is used, for example, in a presentation in which one presenter S1 makes a presentation to a plurality of listeners L1 to L3. Note that FIG. 1 shows, as an example, the case where there are three listeners.

The information processing device 1 is, for example, a terminal used by the presenter S1 and is a desktop personal computer (PC), a notebook PC, a tablet terminal, or a smartphone. The information processing device 1 stores, for example, the manuscript data of the presentation. Further, the information processing apparatus 1 has, for example, a display (not shown) and presents the contents of the manuscript data to the presenter S1.

The display device 3 is, for example, a liquid crystal display, a screen, or the like, and displays the contents of the original data. For example, the display device 3 can communicate with the information processing device 1. Specifically, the information processing device 1 transmits a manuscript image M1 indicating the contents of the manuscript data to the display device 3. The display device 3 receives and displays the original image M1 from the information processing device 1.

For example, in a presentation, it is desirable to record the content of the presenter's presentation and the questions from the listener. On the other hand, a method of recording the presentation by voice and collating the manuscript data can be considered.

Next, the configuration of the information processing apparatus 1 according to the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 2 is a diagram showing an information processing device 1 according to the present embodiment.

The information processing device 1 includes a control unit 11 and a storage unit 12. The control unit 11 includes a voice recognition unit 13, a conversion unit 14, a comparison processing unit 15, and a display processing unit 16. The control unit 11 controls the operation of each unit of the information processing device 1. Specifically, the control unit 11 includes a processor such as a CPU (Central Processing Unit), and controls the operation of each unit of the information processing device 1 by executing, for example, a computer program stored in the storage unit 12. ..

The storage unit 12 stores the manuscript data. The display processing unit 16 performs a process of displaying the original image M1 on the display device 3, for example. Specifically, the display processing unit 16 refers to the storage unit 12 to generate a manuscript image M1 showing the contents of the manuscript data, and transmits the manuscript image M1 to the display device 3.

The voice recognition unit 13 recognizes the voice input via the microphone 2. For example, the voice uttered by the presenter S1 and the listeners L1 to L3 is converted into an electric signal by the microphone 2 and input to the voice recognition unit 13. The voice recognition unit 13 performs signal processing such as A / D conversion and Fourier transform on the electric signal obtained by converting the voice. The voice recognition unit 13 determines the content of the voice based on the frequency, the waveform, and the like obtained as a result of the signal processing.

The conversion unit 14 converts the content of the voice determined by the voice recognition unit 13 into character data.

In the present embodiment, it is distinguished whether the voice input through the microphone 2 is the voice uttered by the presenter S1 or the voice uttered by the listeners L1 to L3.

Next, with reference to FIGS. 2 and 3, a comparison process between the voice and the manuscript data by the information processing apparatus 1 according to the present embodiment will be described. FIG. 3 is a diagram showing an example of comparison processing between voice and manuscript data by the information processing apparatus 1 according to the present embodiment.

In the present embodiment, when the comparison process is performed by the information processing apparatus 1, the presenter S1 is registered before the start of the presentation.

For example, the voice recognition unit 13 recognizes a specific voice and registers it as a speaker. Specifically, when the presenter S1 is registered as a speaker, the presenter S1 utters a predetermined word or sentence into the microphone 2, for example. The uttered voice is converted into an electric signal by the microphone 2 and input to the voice recognition unit 13. The voice recognition unit 13 performs signal processing such as A / D conversion and Fourier transform on the electric signal obtained by converting the voice. The voice recognition unit 13 extracts the characteristics of the voice (V1) uttered by the presenter S1 based on the frequency, the waveform, and the like obtained as a result of the signal processing. The voice recognition unit 13 stores, for example, the characteristics of the voice V1 and the presenter S1 in association with each other in the storage unit 12.

As described above, in the presentation, in the presentation, the voice input through the microphone 2 is the voice uttered by the presenter S1, or is uttered by a person other than the presenter S1 (for example, listeners L1 to L3). It is possible to distinguish whether it is a voice that has been processed.

Specifically, the voice recognition unit 13 performs signal processing such as A / D conversion and Fourier transform on the electric signal converted by the voice V2 uttered by the presenter S1 in the presentation. The voice recognition unit 13 extracts the characteristics of the voice V2 based on the frequency, the waveform, and the like obtained as a result of the signal processing. The voice recognition unit 13 compares the characteristics of the extracted voice V2 with the characteristics of the voice V1 of the storage unit 12. When the characteristics of the extracted voice V2 and the characteristics of the voice V1 of the storage unit 12 match or are completely matched, the voice recognition unit 13 uses the voice uttered by the presenter S1 for the voice V2. Judge that there is.

On the other hand, when the voice V3 other than the voice V2 is input via the microphone 2, the voice recognition unit 13 performs signal processing such as A / D conversion and Fourier transform on the electric signal converted by the voice V3. The voice recognition unit 13 extracts the characteristics of the voice V3 based on the frequency, the waveform, and the like obtained as a result of the signal processing. The voice recognition unit 13 is a voice uttered by a person other than the presenter S1 when the characteristics of the extracted voice V3 and the characteristics of the voice V1 of the storage unit 12 match less than a certain degree or do not match at all. Is determined.

The conversion unit 14 distinguishes between the voice V2 by the presenter S1 registered by the voice recognition unit 13 and the voice V3 by other than the presenter S1 and converts each voice into character data. For example, the conversion unit 14 converts the content of the voice determined by the voice recognition unit 13 into character data, and when the voice recognition unit 13 determines that the voice is the voice V2 uttered by the presenter S1, the announcement is made. Information to the effect that it is the voice of person S1 is added to the character data. Further, when the voice recognition unit 13 determines that the voice is a voice V3 uttered by a voice other than the presenter S1, the conversion unit 14 adds information to the effect that the voice is a voice other than the presenter S1 to the character data.

The comparison processing unit 15 compares the character data converted by the conversion unit 14 with the manuscript data, and when the character data and the manuscript data match, changes the property of the corresponding character in the manuscript data. In the example of FIG. 3, when the manuscript data contains the same characters as the character data “ABC” converted by the conversion unit 14, the comparison processing unit 15 has the color, size, and font of the characters “ABC” included in the manuscript data. And at least one of them is changed. For example, the comparison processing unit 15 changes the color, size, font, etc. of the same character "ABC" as the character data "ABC" converted by the conversion unit 14 to a color, size, font, etc. different from other characters in the manuscript data. Change each.

In the present embodiment, the comparison processing unit 15 includes characters corresponding to the character data converted by the voice V2 of the presenter S1 in the manuscript data, and characters corresponding to the character data converted by the voice V3 by a voice V3 other than the presenter S1. To a different property.

Next, with reference to FIGS. 3 and 4, the property change of the characters included in the manuscript data by the information processing apparatus 1 according to the present embodiment will be described. FIG. 4 is a diagram showing an example of changing the property of characters included in the manuscript data by the information processing apparatus 1 according to the present embodiment.

FIG. 3 shows a case where the presenter S1 emits the voice “ABC” in the presentation. FIG. 4 shows a case where the listener L2 emits the voice “not ABC” in the presentation.

The voice recognition unit 13 performs signal processing such as A / D conversion and Fourier transform on the electric signal converted by the voice "ABC" uttered by the presenter S1 in the presentation. The voice recognition unit 13 extracts the characteristics of the voice "ABC" based on the frequency, the waveform, and the like obtained as a result of the signal processing. The voice recognition unit 13 compares the characteristics of the extracted voice "ABC" with the characteristics of the voice V1 of the storage unit 12. The voice recognition unit 13 utters the voice "ABC" by the presenter S1 when the characteristics of the extracted voice V2 and the characteristics of the voice V1 of the storage unit 12 match or completely match. Judged as voice.

The conversion unit 14 converts the voice "ABC" determined by the voice recognition unit 13 into the character data "ABC", and adds information to the effect that it is the voice of the presenter S1 to the character data "ABC".

The comparison processing unit 15 compares the character data "ABC" converted by the conversion unit 14 with the manuscript data, and changes the property of the character "ABC" included in the manuscript data to the property P1. For example, the property P1 is that the color of the character "ABC" is different from that of other characters in the manuscript data. Further, for example, if the color of the character "ABC" is changed to the same color as the background, the character "ABC" can be made invisible.

The display processing unit 16 generates a manuscript image M2 showing the contents of the manuscript data after the property is changed by the comparison processing unit 15, and transmits the manuscript image M2 to the display device 3. The display device 3 receives and displays the original image M2 from the information processing device 1.

On the other hand, in the example of FIG. 4, when the voice "not ABC" emitted by the listener L2 is input through the microphone 2, the voice recognition unit 13 responds to the electric signal to which the voice "not ABC" is converted. It performs signal processing such as A / D conversion and Fourier transform. The voice recognition unit 13 extracts the characteristics of the voice "not ABC" based on the frequency, the waveform, and the like obtained as a result of the signal processing. If the characteristics of the extracted voice "not ABC" and the characteristics of the voice V1 of the storage unit 12 match less than a certain degree or do not match at all, the voice recognition unit 13 announces the voice "not ABC" S1. It is determined that the voice is uttered by other than.

The conversion unit 14 converts the voice "not ABC" determined by the voice recognition unit 13 into the character data "not ABC", and adds information to the effect that the voice is other than the presenter S1 to the character data.

The comparison processing unit 15 searches whether a part or all of the character data “not ABC” converted by the conversion unit 14 is included in the manuscript data. The comparison processing unit 15 determines that "ABC" is included in the manuscript data, and changes the property of the character "ABC" included in the manuscript data to a property P2 different from the property P1. For example, the property P2 is such that the color of the character "ABC" is different from other characters in the manuscript data and the property P1.

The display processing unit 16 generates a manuscript image M3 showing the contents of the manuscript data after the property is changed by the comparison processing unit 15, and transmits the manuscript image M3 to the display device 3. The display device 3 receives and displays the original image M3 from the information processing device 1.

Next, with reference to FIG. 5, the re-change of the character property by the information processing apparatus 1 according to the present embodiment will be described. FIG. 5 is a diagram showing an example of re-changing character properties by the information processing apparatus 1 according to the present embodiment.

For example, when the voice by the presenter S1 satisfies a predetermined condition, the comparison processing unit 15 re-changes the character property P2 corresponding to the converted character data by the voice other than the presenter S1 to the property P1.

For example, when the presenter S1 emits a voice V4 including the words "correction" or "examination" and the word "ABC" after the property of the character included in the manuscript data is changed to the property P2 by the comparison processing unit 15. , The property is changed again.

Specifically, after the property of the character included in the manuscript data is changed to the property P2 by the comparison processing unit 15, when the voice V4 emitted by the presenter S1 is input via the microphone 2, the voice recognition unit 13 Performs signal processing such as A / D conversion and Fourier conversion on the electric signal converted by the voice V4.

The voice recognition unit 13 extracts the characteristics of the voice V4 based on the frequency, waveform, etc. obtained as a result of signal processing. The voice recognition unit 13 compares the characteristics of the extracted voice V4 with the characteristics of the voice V1 of the storage unit 12. When the characteristics of the extracted voice V4 and the characteristics of the voice V1 of the storage unit 12 match or are completely matched, the voice recognition unit 13 uses the voice uttered by the presenter S1 for the voice V4. Judge that there is.

The conversion unit 14 converts the voice V4 determined by the voice recognition unit 13 into character data, and adds information to the effect that it is the voice of the presenter S1 to the character data.

The comparison processing unit 15 compares the words "correction" and "ABC" included in the character data converted by the conversion unit 14 with the manuscript data, and changes the property of the character "ABC" included in the manuscript data to the property P1. ..

The display processing unit 16 generates a manuscript image M4 showing the contents of the manuscript data after the property is changed by the comparison processing unit 15, and transmits the manuscript image M4 to the display device 3. The display device 3 receives and displays the original image M4 from the information processing device 1.

In this way, by distinguishing between the utterances by the presenter S1 and the utterances by other than the presenter S1 and changing the corresponding characters in the manuscript data to different properties, items to be confirmed after the presentation or during the presentation were solved. It becomes easy to confirm the matter.

In the present embodiment, if the voice input via the microphone 2 satisfies a predetermined condition, the voice may not be converted into character data.

For example, as a result of signal processing by the voice recognition unit 13 with respect to the converted electric signal of the voice input via the microphone 2, the voice input through the microphone 2 is a voice having a predetermined volume or less, or a predetermined voice. When it is determined that the voice is the tone of the above, the conversion unit 14 does not convert the content of the voice determined by the voice recognition unit 13 into character data.

Next, the comparison process according to the present embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing a comparison process according to the present embodiment.

First, the voice recognition unit 13 recognizes the voice produced by the presenter S1 and registers it as a speaker (step S11).

The voice recognition unit 13 performs signal processing such as A / D conversion and Fourier transform on the converted electric signal of the voice input via the microphone 2 (step S12).

As a result of signal processing, the voice recognition unit 13 determines whether the voice input via the microphone 2 has a sufficient volume (step S13). When the voice recognition unit 13 determines that the voice input via the microphone 2 is not sufficiently loud (No in step S13), the conversion unit 14 displays the content of the voice determined by the voice recognition unit 13 as characters. It waits until a new voice is input without converting it into data (step S12).

On the other hand, when the voice recognition unit 13 determines that the voice input via the microphone 2 is sufficiently loud (Yes in step S13), the voice recognition unit 13 announces whether the voice is the voice of the presenter S1. It is determined whether the voice is from a person other than the person S1 (step S14).

When the voice recognition unit 13 determines that the voice input via the microphone 2 is the voice by the presenter S1 (Yes in step S14), the conversion unit 14 determines the content of the voice by the voice recognition unit 13. Is converted into character data (step S15).

The comparison processing unit 15 compares the character data converted by the conversion unit 14 with the manuscript data, and changes the property of the corresponding character included in the manuscript data to the property P1 (step S16). The conversion unit 14 and the comparison processing unit 15 wait until a new voice is input (step S12).

On the other hand, when the voice recognition unit 13 determines that the voice input via the microphone 2 is a voice other than the presenter S1 (No in step S14), the conversion unit 14 is determined by the voice recognition unit 13. The content of the voice is converted into character data (step S17).

The comparison processing unit 15 searches whether a part or all of the character data converted by the conversion unit 14 is included in the manuscript data (step S18).

The comparison processing unit 15 changes the property of the corresponding character included in the manuscript data to the property P2 (step S19).

The conversion unit 14 and the comparison processing unit 15 wait until a new voice is input via the microphone 2. When a new voice is input, the voice recognition unit 13 determines whether the input voice is a voice by the presenter S1 or a voice other than the presenter S1 (step S20).

When the voice recognition unit 13 determines that the voice input via the microphone 2 is a voice other than the presenter S1 (No in step S20), the conversion unit 14 and the comparison processing unit 15 have steps S17 to S19. Is processed.

On the other hand, when the voice recognition unit 13 determines that the voice input via the microphone 2 is a voice other than the presenter S1, the conversion unit 14 and the comparison processing unit 15 use the microphone 2 (Yes in step S20). It is determined whether or not a predetermined word is included in the content of the voice input by the presenter S1 (step S21).

When the content of the voice by the presenter S1 includes a predetermined word (Yes in step S21), the comparison processing unit 15 sets the property P2 of the character corresponding to the character data to which the voice by other than the presenter S1 is converted. Change to P1 (step S16). The conversion unit 14 and the comparison processing unit 15 wait until a new voice is input (step S12).

On the other hand, when the comparison processing unit 15 does not include a predetermined word in the content of the voice by the presenter S1 (No in step S21), the comparison processing unit 15 maintains the property P2 of the corresponding character. The conversion unit 14 and the comparison processing unit 15 wait until a new voice is input (step S12).

In the present embodiment, the number of display devices 3 is one, but the number is not limited to this, and may be, for example, the number of participants in the presentation (four in the example of FIG. 1), and the display device 3 may be used. It does not have to be provided.

In the present embodiment, the information processing apparatus 1 is a terminal used by the presenter S1, but the present invention is not limited to this, and may be, for example, a server provided inside or outside the presentation venue. In this case, the electric signal converted from the voice input via the microphone 2 is transmitted to the information processing apparatus 1. The information processing apparatus 1 receives the transmitted electric signal and performs various processes.

In the present embodiment, the information processing device 1 is configured to input an electric signal converted from the voice input via the microphone 2, but the present invention is not limited to this, and the information processing device 1 is pre-recorded. An electric signal corresponding to the audio data may be input.

In the present embodiment, the manuscript data is stored in the storage unit 12, but the present invention is not limited to this. For example, the information processing device 1 does not have the storage unit 12, and the control unit 11 acquires the manuscript data from the outside. It may be configured to be used.

The embodiments of the present invention have been described above with reference to the drawings (FIGS. 1 to 6). However, the present invention is not limited to the above embodiment, and can be implemented in various embodiments without departing from the gist thereof. The drawings are schematically shown mainly for each component for easy understanding, and the thickness, length, number, etc. of each of the illustrated components are different from the actual ones for the convenience of drawing creation. .. Further, the material, shape, dimensions, etc. of each component shown in the above embodiment are merely examples, and are not particularly limited, and various changes can be made without substantially deviating from the effects of the present invention. be.

The present invention can be used in the field of minutes preparation at a meeting.

1: Information processing device 3: Display device 11: Control unit 12: Storage unit 13: Voice recognition unit 14: Conversion unit 15: Comparison processing unit 16: Display processing units M1 to M4: Original image P1, P2: Property S1: Announcement Person V1 to V4: Voice

Claims (4)

A voice recognition unit that recognizes voice and
A conversion unit that converts the voice recognized by the voice recognition unit into character data, and a conversion unit.
The character data converted by the conversion unit is compared with the manuscript data, and when the character data and the manuscript data match, a comparison processing unit for changing the property of the corresponding character in the manuscript data is provided.
The voice recognition unit recognizes a specific voice and registers it as a speaker.
The conversion unit distinguishes between the voice by the speaker registered by the voice recognition unit and the voice by a person other than the speaker, and converts each of the voices into the character data.
The comparison processing unit differs between the character corresponding to the character data in which the voice by the speaker in the manuscript data is converted and the character corresponding to the character data in which the voice by a person other than the speaker is converted. An information processing device that changes to the above property. When the voice by the speaker satisfies a predetermined condition, the comparison processing unit sets the property of the character corresponding to the character data converted by the voice other than the speaker to the voice by the speaker in the manuscript data. The information processing apparatus according to claim 1, wherein the property is changed to the same property as the character corresponding to the converted character data. The information processing device according to claim 1 or 2, wherein the conversion unit does not convert the voice into character data when the voice satisfies a predetermined condition. The information processing device is
The information processing device according to any one of claims 1 to 3, further comprising a display processing unit that performs a process of displaying an image corresponding to the manuscript data on the display device.

Images