JP2016139972A - Information processing unit, voice output method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a user's judgement easier with a voice output.SOLUTION: The information processing unit includes: a main control section for obtaining a voice output parameter in accordance with an image display position associated with voice to be output; and a voice control section for outputting the voice to be output in accordance with the output parameter to a voice output apparatus. The output parameter is a parameter for changing a position of an output source of the voice recognized by a user listening to the voice.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for outputting sound.

  Conventionally, techniques for outputting sound have been used in various fields. For example, a technique for outputting sound in accordance with image output has been proposed (see, for example, Patent Document 1). More specifically, there is a voice chat technique that enables a conversation with an image with another user located in a remote place.

Japanese Patent No. 5478357

  However, in the voice chat, there is a problem that when talking simultaneously with a plurality of other users, the voices of a plurality of people are mixed and difficult to hear. In this way, depending on only the voice that is simply output, the user's judgment to be performed along with the voice (for example, judgment as to who is speaking among a plurality of people displayed on the screen) ) May be delayed or difficult to judge.

  In view of the circumstances described above, an object of the present invention is to provide a technique that makes it easier for a user to make a judgment by outputting sound.

  According to one aspect of the present invention, a main control unit that acquires a sound output parameter according to a display position of an image related to a sound to be output, and outputs the sound to be output to a sound output device according to the output parameter And an audio control unit that controls the output parameter to change a position of an output source of the audio recognized by a user who listens to the audio.

  One embodiment of the present invention is the above information processing device, in which the voice is a voice representing a utterance content of a speaker different from the user, and the image is an image showing the speaker.

  One aspect of the present invention is the information processing apparatus, wherein the voice is a voice uttered by a speaker who interacts with the user via a network.

  One aspect of the present invention is the information processing apparatus, wherein the main control unit acquires different output parameters for a plurality of sounds, and the sound control unit outputs the plurality of sounds simultaneously.

  One aspect of the present invention is the information processing apparatus, wherein the main control unit obtains an audio output parameter according to a display position of an image to be monitored in which an abnormality has occurred.

  One aspect of the present invention is the information processing apparatus, wherein the main control unit obtains an audio output parameter in accordance with a display position of an image to which a user should pay attention.

  According to one aspect of the present invention, an acquisition step of acquiring an audio output parameter corresponding to a display position of an image related to an output target audio, and causing the audio output device to output the output target audio according to the output parameter An output audio control step, wherein the output parameter is a parameter for changing a position of an output source of the audio recognized by a user who listens to the audio.

  According to one aspect of the present invention, a main control unit that acquires a sound output parameter according to a display position of an image related to a sound to be output, and outputs the sound to be output to a sound output device according to the output parameter A computer for causing the computer to function as an information processing apparatus, wherein the output parameter is a parameter for changing a position of an output source of the audio recognized by a user who listens to the audio It is a program.

  According to the present invention, it is possible to make a user's judgment easier by outputting sound.

1 is a system configuration diagram illustrating a system configuration of an audio output system 1. FIG. 5 is a flowchart illustrating a first specific example of the flow of output processing of audio data by the main control unit 301. 12 is a flowchart illustrating a second specific example of the flow of output processing of audio data by the main control unit 301. It is a figure which shows the specific example of the screen of the image output apparatus 20 of a television conversation system. It is a figure which shows the specific example of an output position information table. It is a figure which shows the specific example of an output parameter table. It is a figure which shows the specific example of the screen of the image output device 20 of a monitoring system. It is a figure which shows the specific example of an output parameter table. It is a figure which shows the specific example of the screen of the image output apparatus 20 of an audio | voice monitoring system. It is a figure which shows the specific example of an output parameter table. It is a figure which shows the specific example of the WEB site displayed in a WEB system.

FIG. 1 is a system configuration diagram showing a system configuration of the audio output system 1. The audio output system 1 includes an audio output device 10, an image output device 20, and a control device 30.
The audio output device 10 is a device that outputs audio such as a speaker.
The image output device 20 is a device that outputs an image such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro Luminescence) display.

The control device 30 acquires audio data indicating the sound to be output and image data indicating the image to be output. The control device 30 causes the audio output device 10 to output the acquired audio data. The control device 30 causes the image output device 20 to output the acquired image data.
When outputting the audio data to the audio output device 10, the control device 30 generates directivity according to a predetermined condition. The sound directivity indicates that the position of a sound output source (hereinafter referred to as “virtual output source”) recognized by a user who listens to the sound can be changed according to a predetermined condition. The directivity of sound can be generated, for example, by outputting the same sound at different timings (with a delay) in a plurality of sound output devices located apart from each other. For example, directivity can be generated in the sound by outputting the same sound at different timings in a plurality of speakers connected (or provided) to one personal computer. The control device 30 changes the directivity of the sound according to a predetermined condition. In other words, the control device 30 changes the position of the virtual output source according to a predetermined condition.

The control device 30 will be described in detail.
The control device 30 includes a CPU (Central Processing Unit), a memory, an auxiliary storage device, and the like connected by a bus, and executes a control program. The control device 30 functions as a device including the main control unit 301, the audio control unit 302, and the display control unit 303 by executing the control program. All or some of the functions of the control device 30 may be realized by using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA).

  The control program may be recorded on a computer-readable recording medium. The computer-readable recording medium is a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, a semiconductor storage device (for example, SSD: Solid State Drive), a hard disk built in a computer system, or a semiconductor storage. A storage device such as a device. The control program may be transmitted via a telecommunication line.

  The main control unit 301 acquires audio data and image data. The main control unit 301 further acquires attribute information of the acquired audio data. The attribute information is information indicating audio attributes. The attribute information includes, for example, a user ID indicating the voice speaker, a device ID indicating the device with which the voice is associated, an item ID indicating an item in the WEB page with which the voice is associated, and a window with which the voice is associated. Relative position.

  The main control unit 301 acquires output position information based on the attribute information. The output position information is information related to the virtual output source. The output position information is, for example, information indicating two-dimensional coordinates in the screen of the image output apparatus 20 and information indicating three-dimensional coordinates in a three-dimensional image displayed on the image output apparatus 20.

  The main control unit 301 acquires an output parameter based on the output position information. The output parameter is information for controlling the audio output device 10 so that the user can hear the audio from the virtual output source according to the output position information. The output parameter and the output position information may be stored in advance by the main control unit 301 in a one-to-one correspondence.

The audio control unit 302 causes the audio output device 10 to output the audio data acquired by the main control unit 301 with the output parameters acquired by the main control unit 301.
The display control unit 303 causes the image output device 20 to output the image data acquired by the main control unit 301.

  FIG. 2 is a flowchart showing a first specific example of the flow of output processing of audio data by the main control unit 301. The main control unit 301 acquires attribute information of audio data that is an output target (step S101). Next, the main control unit 301 acquires an output parameter based on the attribute information acquired in step S101 (step S102). Then, the main control unit 301 instructs the audio control unit 302 to output the audio data to be output with the output parameter acquired in step S102 (step S103).

  FIG. 3 is a flowchart showing a second specific example of the flow of audio data output processing by the main control unit 301. The main control unit 301 acquires attribute information of audio data that is an output target (step S111). Next, the main control unit 301 acquires output position information based on the attribute information acquired in step S111 (step S112). Next, the main control unit 301 acquires an output parameter based on the output position information acquired in step S112 (step S113). Then, the main control unit 301 instructs the audio control unit 302 to output the audio data to be output with the output parameter acquired in step S113 (step S114).

  In the first specific example, attribute information and output parameters are stored in the main control unit 301 in advance in association with each other. The first specific example is a process used when the sound directivity (output source) of each attribute is fixedly determined.

In the second specific example, attribute information and output position information are stored in the main control unit 301 in association with each other. Further, output position information and output parameters are stored in the main control unit 301 in association with each other. In the second specific example, the relationship between the attribute information and the output parameters can be easily changed by changing the association between the attribute information and the output position information. The second specific example is a process used when the sound directivity (output source) of each attribute changes.
Next, a specific application example of the audio output system 1 will be described.

[First application example: TV conversation system]
An audio output system 1 applied to a television conversation system will be described. The audio data that is output is audio that is spoken by a conversation person who interacts via a network with a user who listens to the audio output by the television conversation system. The output image data is an image showing a conversation person.

  FIG. 4 is a diagram showing a specific example of the screen of the image output device 20 of the television conversation system. In the example of FIG. 4, the user is interacting with four interlocutors. Images 401 to 404 showing a plurality of interlocutors are displayed on the screen. In the example of FIG. 4, as the images 401 to 404, moving images obtained by copying each interactor's face with a camera are displayed. The images displayed in the images 401 to 404 are not limited to the moving images of the faces of the respective interactors. For example, it may be a still image of each interactor's face or a character indicating each interactor.

  FIG. 5 is a diagram showing a specific example of the output position information table. The output position information table shown in FIG. 5 is stored by the main control unit 301. The output position information table has a plurality of records in which user IDs and output position information are associated with each other. The user ID is identification information of a conversation person. The television conversation system has the user ID of each conversation person as attribute information of voice data representing the utterance content of each conversation person. The output position information indicates a position on the screen of the image output device 20 at which an image of the conversation person indicated by the user ID is displayed.

  For example, the top record in FIG. 5 indicates that the moving image of the face of the conversation person whose user ID is “A1” is displayed at the upper left position of the screen (the position of the image 401 in FIG. 4). The second record from the top in FIG. 5 indicates that the moving image of the face of the conversation person whose user ID is “A2” is displayed at the upper right position of the screen (the position of the image 402 in FIG. 4). The third record from the top in FIG. 5 indicates that the moving image of the face of the conversation person whose user ID is “A3” is displayed at the lower left position of the screen (the position of the image 403 in FIG. 4). The fourth record from the top in FIG. 5 indicates that the moving image of the face of the conversation person whose user ID is “A4” is displayed at the lower right position of the screen (the position of the image 404 in FIG. 4).

  The main control unit 301 obtains voice data representing the utterance content of each interlocutor by receiving it via the network. The voice data is received together with identification information indicating the interlocutor. The identification information received together with the voice data may be, for example, an IP address assigned to a device operated by a dialog person, or may be a MAC address of the device. The main control unit 301 acquires the user ID (attribute information) of the conversation person based on the acquired identification information. The main control unit 301 refers to the output position information table to acquire output position information corresponding to the user ID of the conversation person.

  FIG. 6 is a diagram illustrating a specific example of the output parameter table. The output parameter table shown in FIG. 6 is stored by the main control unit 301. The output parameter table has a plurality of records in which output position information and output parameters are associated with each other. The output parameter is a parameter for controlling the position of the sound output source recognized by the user who listens to the sound. That is, the output parameter is a parameter for giving the voice directivity such that the position indicated by the output position information is the voice output source. The output parameter may be defined as the output timing of the audio output devices 10 arranged on the left and right, for example.

  For example, the top record in FIG. 6 indicates that the output parameter is Pa1 when the output position information of the audio data to be output is at the upper left. The second record from the top in FIG. 6 indicates that the output parameter is Pa2 when the output position information of the audio data to be output is on the upper right. The third record from the top in FIG. 6 indicates that the output parameter is Pa3 when the output position information of the audio data to be output is in the lower left. The fourth record from the top in FIG. 6 indicates that the output parameter is Pa4 when the output position information of the audio data to be output is in the lower right.

The main control unit 301 acquires an output parameter corresponding to the output position information of the audio data to be output by referring to the output parameter table.
In the television conversation system configured as described above, sound data is output with directivity corresponding to the position where the image of the conversation person is displayed. For example, on the screen of FIG. 4, sound representing the utterance content of the conversation person in the image 401 is output as if there is an output source in the upper left direction (for example, the position of the image 401 on the screen). In the screen of FIG. 4, the sound representing the utterance content of the conversation person in the image 402 is output as if there is an output source in the upper right direction (for example, the position of the image 402 on the screen). The following effects can be obtained by outputting the sound in this way.

  Generally, when voices spoken by a plurality of people at the same time are output from a single output source (that is, output as monaural speech), it is difficult for people to hear them. Therefore, particularly when audio data is generated with monaural audio, a television conversation system often has a one-to-one conversation. On the other hand, as explained in the cocktail party effect, even if the voice is spoken by multiple people at the same time, if the output source of each voice is different, the person can hear the voice he / she wants among them. It is. In the above-described television conversation system, the voices of a plurality of interlocutors are output so that they can be heard from different output sources. Therefore, even if monaural sound is used, the user can hear the sound spoken by a plurality of people at the same time.

  Also, even if the user cannot tell who the voice of the interlocutor is, the voice is output from the output source corresponding to the position of the image displayed on the screen. It is possible to determine which voice of the conversation person.

<Modification>
The identification information received together with the audio data may be a user ID indicating a conversation person.
The number of interlocutors is not limited to four. For example, there may be three people or five or more people.
The present invention may be applied as a conversation system that does not involve image output without using the image output device 20. Even in the case of such a configuration, it is possible to obtain an effect that the user can hear the voices spoken by a plurality of people at the same time.

[Second application example: Monitoring system]
The audio output system 1 applied to the monitoring system will be described. The output audio data is an alarm sound indicating that an abnormality has occurred. There may be one type of audio data or a plurality of types. The output image data is an image indicating a monitoring target. The monitoring target may be a device or a facility. The monitoring system acquires a state value from a sensor provided in the monitoring target, and determines whether an abnormality has occurred based on the state value. When an abnormality occurs, a sound corresponding to the monitoring target where the abnormality has occurred is output.

  FIG. 7 is a diagram illustrating a specific example of the screen of the image output apparatus 20 of the monitoring system. In the example of FIG. 7, there are eight devices to be monitored, and each device is connected to one or more other devices via a communication line. On the screen, images 501 to 508 indicating a plurality of monitoring targets are displayed. In the example of FIG. 7, images representing the respective monitoring targets are displayed as the images 501 to 508. The images displayed as the images 501 to 508 may be different from the image displayed when it is determined to be normal and the image displayed when it is determined that an abnormality has occurred.

  FIG. 8 is a diagram showing a specific example of the output parameter table. The output parameter table shown in FIG. 8 is stored by the main control unit 301. The output parameter table has a plurality of records in which monitoring target IDs are associated with output parameters. The monitoring target ID is identification information of the monitoring target. The monitoring system has a monitoring target ID of each monitoring target as attribute information of audio data output when an abnormality occurs. Note that, when a plurality of monitoring targets share the same sound output when an abnormality occurs, a plurality of attribute information may be assigned to one sound data.

  In the monitoring system according to the present embodiment, the position in the screen on which each monitoring target image is displayed is fixedly defined. Therefore, the position of the sound output source that is output when an abnormality occurs in each monitoring target does not change. Therefore, the monitoring system in this embodiment can be implemented by the process of the first specific example without using the output position information.

  For example, when there are a large number of monitoring targets and it is difficult to search for a changed image, or when the screen of the image output device 20 is very large and it is difficult to visually recognize the entire screen at once, the user Even if a sound indicating the occurrence of an abnormality is output, it cannot be easily determined in which monitoring object the abnormality has occurred. In response to such a problem, in the monitoring system of the present embodiment, sound indicating abnormality is output from an output source corresponding to the position of the image to be monitored displayed on the screen. Therefore, the user can more easily and earlier determine in which monitoring target an abnormality has occurred.

<Modification>
You may comprise so that the position within the screen where the image of each monitoring object is displayed changes. In this case, a table in which the identification information of each monitoring target is associated with the output position information is required. Such a table may be acquired from another information processing apparatus, may be set by a user, or may be acquired by another method.

[Third application example: Voice monitoring system]
The audio output system 1 applied to the audio monitoring system will be described. The output voice data is voice uttered by the speaker who is the subject of monitoring. The output image data is an image showing a speaker.
FIG. 9 is a diagram illustrating a specific example of the screen of the image output device 20 of the audio monitoring system. In the example of FIG. 9, 16 speakers are monitored. Images 601 to 616 indicating a plurality of speakers are displayed on the screen. In the example of FIG. 9, as the images 601 to 616, moving images obtained by copying each speaker's face with a camera are displayed. The images displayed in the images 601 to 601 are not limited to the moving images of the faces of the speakers. For example, it may be a still image of the face of each speaker or a character indicating each speaker.

  FIG. 10 is a diagram illustrating a specific example of the output parameter table. The output parameter table shown in FIG. 10 is stored by the main control unit 301. The output parameter table has a plurality of records in which operator IDs and output parameters are associated with each other. The operator ID is operator identification information. The voice monitoring system has an operator ID of each operator as attribute information of the voice data to be output.

  In the voice monitoring system according to the present embodiment, the position in the screen where the image of each operator is displayed is fixedly defined. Therefore, the position of the voice output source of each operator does not change. Therefore, the voice monitoring system in the present embodiment can be implemented by the process of the first specific example without using the output position information.

  The main control unit 301 outputs a predetermined number of voices of a plurality of speakers simultaneously with different directivities. The main control unit 301 changes the speaker to whom the sound is output at a predetermined timing (for example, 10 seconds after the sound is output). For example, at the first timing, the main control unit 301 simultaneously outputs audio data of the utterance contents of the first speaker combination (for example, the speakers of the images 601 to 604). When a predetermined time (for example, 10 seconds) elapses from the start of the output of the audio data, the main control unit 301 simultaneously outputs the audio data of the utterance contents of the next speaker combination (for example, the speakers of the images 605 to 608). When a predetermined time has elapsed since the start of the output of the audio data, the main control unit 301 simultaneously outputs the audio data of the utterance content of the next speaker combination (for example, the speakers of the images 609 to 612). By repeating such processing by the main control unit 301, it becomes possible to monitor the utterance contents of all the speakers.

  Note that the main control unit 301 may display the image of the speaker from which the speech data of the utterance content is output in a mode different from the image of the speaker from which the speech data of the utterance content is not output. For example, the main control unit 301 may display an image of a speaker to which voice data of the utterance content is output with a thick frame.

  For example, when there are a large number of operators, if the voice is monitored one by one, it takes a lot of time to complete the monitoring of the voices of all the members. In response to such a problem, in the voice monitoring system of the present embodiment, voices of a plurality of speakers are output simultaneously. At that time, each sound is output so that it can be heard from different output sources depending on the position of the image. Therefore, the user can monitor while listening to a plurality of voices simultaneously by the cocktail party effect described above. Therefore, it is possible to shorten the time until the monitoring of all members is completed.

<Modification>
The position on the screen where the image of each operator is displayed may be changed. In this case, a table in which the identification information of each operator is associated with the output position information is required. Such a table may be acquired from another information processing apparatus, may be set by a user, or may be acquired by another method.

[Fourth application example: WEB system]
The audio output system 1 applied to the WEB system will be described. The output audio data is audio data defined in advance on the WEB site. The output image data is an image defined in advance on the WEB site.

  The main control unit 301 acquires the position in the screen of the image associated with the output sound as the attribute information. The main control unit 301 acquires an output parameter according to the acquired position in the screen. The relationship between the position in the screen and the value of the output parameter may be defined in advance by an equation. In this case, the main control unit 301 stores in advance a mathematical expression representing the relationship between the position in the screen and the value of the output parameter.

  FIG. 11 is a diagram showing a specific example of a WEB site displayed in the WEB system. In this WEB site, an input frame 701 for inputting a name, an input frame 702 for inputting an address, and an input frame 703 for inputting an occupation are displayed. When there is an error in the input content, the main control unit 301 acquires position information indicating the position on the screen of the input frame in which the input content is incorrect. The main control unit 301 acquires the position information of the input frame as follows, for example. First, the main control unit 301 acquires position information of the input frame in the window 70 indicating information on the WEB site (for example, position information of the upper left end of the input frame with the upper left end of the window 70 as the origin). Next, the main control unit 301 acquires the position information of the window 70 in the screen of the image output device 20 (for example, the coordinates of the position of the upper left corner of the window 70 in the screen). The main control unit 301 acquires position information of the input frame in the screen based on the acquired two pieces of information.

  The main control unit 301 acquires the value of the output parameter based on the acquired position information (attribute information). If there is an error in the input frame 701 for inputting the name, an error sound is output so that the place where the input frame 701 is displayed at that time becomes the output source. If there is an error in the input frame 702 for inputting an address, an error sound is output so that the location where the input frame 702 is displayed at that time is the output source. If there is an error in the input box 703 for inputting the occupation, an error sound is output so that the place where the input box 703 is displayed at that time becomes the output source.

  An example of the output sound is an error sound associated with the input frame, but the output sound is not limited to this example. For example, audio associated with a still image of an advertisement (for example, an advertisement banner) or a moving image of an advertisement may be output. In this case, the sound of the advertisement is output so that the sound can be heard from the output source of the position where the image of the advertisement is displayed. For example, sounds associated with images such as buttons included in the tutorial may be output. In this case, for example, the sound associated with the button is output so that the sound can be heard using the position where the image of the button to be operated next is displayed as the output source.

  In the WEB system configured as described above, even when a large amount of information and images are displayed on the WEB site, the sound is output using the position where the user should pay attention as an output source. Therefore, natural line-of-sight guidance to a position where attention should be directed to the user is possible.

<Modification>
The output sound and image are not limited to WEB sound and screen. For example, by applying to a digital signage system, audio and images related to advertisements may be output. For example, the above-described audio and image output may be performed by a tutorial created as an application instead of WEB.
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Audio | voice output system, 10 ... Audio | voice output apparatus, 20 ... Image output apparatus, 30 ... Control apparatus, 301 ... Main control part, 302 ... Audio | voice control part, 303 ... Display control part

Claims (8)

A main control unit that acquires an output parameter of sound according to a display position of an image related to the sound to be output;
A voice control unit that causes the voice output device to output the voice to be output according to the output parameter, and
The information processing apparatus, wherein the output parameter is a parameter for changing a position of an output source of the sound recognized by a user who listens to the sound. The voice is a voice representing the utterance content of a speaker different from the user,
The information processing apparatus according to claim 1, wherein the image is an image showing the speaker.   The information processing apparatus according to claim 2, wherein the voice is a voice uttered by a speaker who interacts with the user via a network. The main control unit acquires different output parameters for a plurality of sounds,
The information processing apparatus according to claim 1, wherein the voice control unit outputs the plurality of voices simultaneously.   The information processing apparatus according to claim 1, wherein the main control unit acquires an audio output parameter corresponding to a display position of an image to be monitored in which an abnormality has occurred.   The information processing apparatus according to claim 1, wherein the main control unit acquires an audio output parameter in accordance with a display position of an image that should be paid attention to the user. An acquisition step of acquiring an audio output parameter according to a display position of an image related to the output target audio;
An output audio control step for causing the audio output device to output the audio to be output according to the output parameter, and
The output parameter is a parameter for changing a position of an output source of the sound recognized by a user who listens to the sound.
Audio output method. A main control unit that acquires an output parameter of sound according to a display position of an image related to the sound to be output;
A voice control unit that causes the voice output device to output the voice to be output according to the output parameter, and
The computer program for causing a computer to function as an information processing apparatus, wherein the output parameter is a parameter for changing a position of an output source of the sound recognized by a user who listens to the sound.

Images