JP5493551B2 - Information processing system, information processing apparatus, and information processing method - Google Patents

Description

  The present invention relates to an information processing system, an information processing apparatus, and an information processing method, and more particularly, to an information processing system, an information processing apparatus, and an information processing method capable of voice control.

  2. Description of the Related Art In recent years, systems such as videophones and videoconferences have been used that can connect remote bases via a network and communicate with remote parties using video and audio.

  In particular, when such a system is used by a plurality of people, the following problems may occur. For example, since the distance from the microphone that acquires the voice, the angle, and the volume of the individual voice are different, it may be difficult to hear a specific voice. Also, for example, operation sounds such as fan sounds generated from devices such as projectors, and operation sounds generated in response to human actions such as PC (Personal Computer) type sounds are collected more than necessary. Sometimes it ends up. Therefore, in a system in which communication is performed by a plurality of people, voice control is important.

  In view of this, a method has been proposed in which voices from various sound sources are separately acquired, so that the separately acquired voices can be individually controlled. For example, voices can be acquired for each person by attaching a pin microphone individually. Further, for example, Patent Document 1 proposes a chair that allows voices to be separated and acquired for each person without embedding a pin microphone by embedding a large number of microphones in the chair. For example, Patent Document 2 proposes an audio conference system that separates and acquires audio and emits sound according to the seating position.

JP 2007-336010 A JP 2008-17126 A

  However, even if the sound can be obtained separately for each sound source, when adjusting the sound, it is necessary to specify which sound source from which sound source is acquired with which microphone. There was a problem that it was difficult.

  Therefore, the present invention has been made in view of the above problems, and the object of the present invention is to specify and control the sound to be controlled from the operation on the screen for displaying the video. It is an object to provide a new and improved information processing apparatus and information processing method.

  In order to solve the above problems, according to an aspect of the present invention, a plurality of microphones, an imaging device that captures images, a sound processing device that processes sound acquired by the plurality of microphones, the images, and An information processing device that outputs sound processed by the sound processing device and outputs sound control information to the sound processing device, wherein the information processing device includes a display unit that displays video; and A coordinate input unit for inputting a coordinate position on the display screen of the display unit, and an audio adjustment interface overlaid on the video in the vicinity of the coordinate position input by the coordinate input unit, displayed on the display unit, and stored internally or externally Based on the correspondence information stored in the unit, the microphone identifier of the microphone that acquires the sound at the position corresponding to the coordinate position in the video is specified, and the specified microphone Besshi and and a control unit for outputting sound control information including a sound processing information corresponding to the input from the coordinate input unit with respect to the audio adjustment interface, the information processing system is provided.

  According to this configuration, the sound acquired by the plurality of microphones is processed by the sound processing device and output by the information processing device. The information processing device outputs video on the display unit, and outputs audio processed by the audio processing device in the audio output unit. Further, the information processing apparatus can output voice control information to the voice processing apparatus. At this time, when the user inputs an object whose sound is to be controlled using the coordinate input unit while viewing the video displayed on the display unit of the information processing device, the information processing device displays a voice adjustment interface at the coordinate position. . Then, the information processing apparatus identifies the microphone to be controlled from the coordinate position using the correspondence information. When the user operates the voice adjustment interface, the information processing apparatus creates voice control information based on the operation information and the identified microphone identifier of the microphone, and outputs the voice control information to the voice processing apparatus. As a result, the user can execute voice control by an intuitive operation while watching the video without considering the correspondence between the operation and the microphone to be controlled.

  In addition, a sound source identifier for identification may be assigned to the sound source in advance, and the correspondence information may be information generated by collecting the correspondence relationship between the sound source identifier and the microphone identifier.

  The plurality of microphones are incorporated in a sound acquisition device, and the sound acquisition device includes a receiver that receives the sound source identifier, and the sound acquisition device is identified on a surface of the sound acquisition device. A recognition code embedded with a voice acquisition device identifier for the display, and the correspondence information includes a correspondence relationship between the voice acquisition device identifier, the sound source identifier, a receiver identifier assigned to the receiver, and the microphone identifier. The information is collected and generated, and the control unit reads the voice acquisition device identifier from the recognition code in the video, and uses the read voice acquisition device identifier and the correspondence information to read the coordinate position. You may specify the microphone identifier of the microphone which acquires the audio | voice of the corresponding position.

  The voice acquisition device has a desk shape, the receiver is installed on a side surface of the voice acquisition device, and the receiver receives the sound source identifier from a sound source identifier transmitter worn by a person. May be received.

  The voice acquisition device further includes a transmitter for transmitting its own transmitter identifier, and the transmitter is opposed to a receiver of the other voice acquisition device when combined with the other voice acquisition device. The receiver receives either the sound source identifier or the transmitter identifier, acquires correspondence information of the plurality of sound acquisition devices, and recognizes the arrangement of the plurality of sound acquisition devices. You may further have an information preparation apparatus.

  The voice processing device processes the voice based on a microphone identifier giving unit that gives a microphone identifier to the voice input from the microphone, a microphone identifier given to the voice, and the voice control information. And an audio mixer.

  The correspondence information may be information in which a correspondence relationship between the coordinate position and the voice acquisition unit identifier is stored in advance.

  In order to solve the above-described problem, according to another aspect of the present invention, an information processing apparatus that acquires and outputs audio and video acquired by a plurality of microphones, the display unit displaying the video A coordinate input unit that inputs a coordinate position on the display screen of the display unit, and a voice adjustment interface that is superimposed on the video and displayed on the display unit in the vicinity of the coordinate position input by the coordinate input unit. Based on correspondence information stored in an external storage unit, a microphone identifier of a microphone that acquires sound at a position corresponding to the coordinate position in the video is specified, and the microphone identifier specified and the voice adjustment interface There is provided an information processing apparatus including a control unit that outputs voice control information including voice processing information according to an input from a coordinate input unit.

  In order to solve the above problem, according to another aspect of the present invention, a display unit that displays an image, a coordinate input unit that inputs a coordinate position on the display screen of the display unit, and a control unit are provided. The information processing apparatus has a method in which the control unit causes a voice adjustment interface to be superimposed on the video and displayed on the display unit in the vicinity of the coordinate position input by the coordinate input unit, and to be stored in an internal or external storage unit. Based on the stored correspondence information, the microphone identifier of the microphone that acquires the sound at the position corresponding to the coordinate position in the video is specified, and the microphone input from the coordinate input unit for the specified microphone identifier and the voice adjustment interface is specified. There is provided an information processing method for outputting voice control information including voice processing information according to an input.

  As described above, according to the present invention, the sound to be controlled can be specified and controlled from the operation on the screen displaying the video.

It is explanatory drawing which shows an example of the audio | voice adjustment screen of the information processing system which concerns on one Embodiment of this invention. It is explanatory drawing which shows an example of the audio | voice adjustment screen of the information processing system which concerns on one Embodiment of this invention. It is a block diagram which shows the function structure of the information processing system which concerns on the 1st Embodiment of this invention. It is explanatory drawing for demonstrating the area which the information processing system which concerns on the 1st Embodiment of this invention uses in order to specify a microphone. It is a table | surface which shows the correspondence information which consists of a correspondence with a microphone identifier and an area. It is a flowchart explaining the operation | movement of the audio | voice adjustment in the information processing system which concerns on one Embodiment of this invention. It is a sub-flowchart which shows the microphone specific flow of the audio | voice adjustment of the information processing system which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the function structure of the information processing system which concerns on the 2nd Embodiment of this invention. It is explanatory drawing which shows an example of the correspondence information in an audio | voice acquisition apparatus. It is the general-view figure which looked at the audio | voice acquisition apparatus from the upper part. It is the general-view figure which looked at the audio | voice acquisition apparatus from the side. It is explanatory drawing explaining communication with a sound source identifier transmitter and an audio | voice acquisition apparatus. It is explanatory drawing explaining the example of an external appearance structure of a batch holder. It is explanatory drawing which shows an example of the corresponding information acquired in the audio | voice acquisition apparatus. It is explanatory drawing which shows an example of the corresponding information acquired in the audio | voice acquisition apparatus. It is explanatory drawing which shows an example of the corresponding information acquired in the audio | voice acquisition apparatus. It is explanatory drawing which shows an example of the corresponding information acquired in the audio | voice acquisition apparatus. It is explanatory drawing which shows the example of arrangement | positioning of the audio | voice acquisition apparatus recognized in a corresponding | compatible information production apparatus. It is explanatory drawing for demonstrating the audio | voice control of the information processing system which concerns on the 2nd Embodiment of this invention. It is a sub-flowchart which shows the microphone specific flow of the audio | voice adjustment of the information processing system which concerns on the 2nd Embodiment of this invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

(Overview)
First, an outline of an information processing system according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. 1 is an explanatory diagram illustrating an example of a sound adjustment screen of an information processing system according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing an example of a sound adjustment screen of the information processing system according to the embodiment of the present invention.

  FIG. 1 shows a scene in which sound adjustment is performed by performing an input operation from the display unit 115 in the information processing system according to the present embodiment. First, the display unit 115 is a display screen that displays an image of the other party in, for example, a video conference or a video phone. Here, it is assumed that the display screen has a function as a coordinate input device with a touch panel laminated on the top. It is assumed that the user who is viewing the display unit 115 feels that the voice of the participant 40 is difficult to hear. The user touches the display unit 115 so as to point at the participant 40. Then, the information processing apparatus 100 identifies the microphone that has acquired the voice of the participant 40, and displays an operation display for adjusting the sound of the identified microphone on the video. Here, the operation display is the volume adjustment bar 60, but is not limited thereto. For example, any sound volume, timbre, balance, effect, etc. that can be adjusted with respect to the sound may be used.

  Here, as shown in FIG. 2, the user operates the operation display 60 to increase the volume. Then, the information processing apparatus 100 generates and outputs a signal for performing control according to the user operation with respect to the microphone specified earlier.

  As described above, the information processing system according to the present embodiment allows an intuitive operation in which, for example, a person who is currently speaking can be confirmed on the screen while pointing to the person while viewing the screen on which the real-time video is displayed. By this, the audio in the video can be controlled. As a result, it is possible to perform audio adjustment without blocking the flow of the field. Further, the user who adjusts the sound does not need to worry about which person's sound is acquired by which microphone.

(First embodiment)
Next, the functional configuration of the information processing system 100 according to the first embodiment of the present invention for realizing the above functions will be described with reference to FIGS. FIG. 3 is a block diagram showing a functional configuration of the information processing system 100 according to the first embodiment of the present invention. FIG. 4 is an explanatory diagram for describing an area used for specifying a microphone by the information processing system according to the first embodiment of the present invention. FIG. 5 is a table showing correspondence information including correspondence relationships between microphone identifiers and areas.

(Configuration of information processing system 100)
The information processing system 100 according to the present embodiment mainly includes an information processing device 110, a sound processing device 120, a microphone 130, and an imaging device 140. The information processing device 110 is a terminal device that displays the video imaged by the imaging device 140, outputs the sound acquired by the microphone 130, and allows the user to input a sound adjustment operation. The information processing apparatus 110 identifies the operation target microphone from the operation information input by the user, generates control information for the microphone, and inputs the control information to the audio processing apparatus 120. The sound processing device 120 executes control on the sound acquired by the microphone 130 according to the control information input from the information processing device 110. For example, when two-way communication is performed between the remote locations AB, when the user a at the remote location A and the user b at the remote location B perform voice adjustment operations on both sides, the user a performs voice adjustment. An information processing system 100a for operation and an information processing system 100b for user b to adjust sound may be provided in each remote location AB.

  The microphone 130 is a sound acquisition device that acquires sound from a sound source. In the present embodiment, the microphone is a stationary microphone that acquires sound for each of six areas using six microphones 130a to 130f. For example, the microphone 130 may be fixed to a ceiling, a wall, a desk, or the like. The microphone 130 is disposed in the same space as the imaging device 140. That is, the microphone 130 acquires the sound of the space imaged by the imaging device 140. At this time, the microphone 130 may be a sound source separation microphone, a directional microphone, or the like that can separate and acquire desired sound.

  The imaging device 140 is a device for capturing a video. The imaging device 140 captures an image and inputs it to the information processing device 110 via the network.

(Information processing apparatus 110)
The information processing apparatus 110 mainly includes a storage unit 111, a communication unit 112, a coordinate input unit 113, a control unit 114, a display unit 115, and an audio output unit 116.

  The storage unit 111 is a storage device that can store information. For example, magnetic recording media such as HDD (Hard Disk Drive), EEPROM (Electronically Erasable and Programmable Read Only Memory), flash memory, MRAM (Magnetoresistive Random Access Memory), FeRAM (Ferroelectric Random Access Memory), PRAM (Phase change Random Access) Non-volatile memory such as “Memory”, but is not limited to the above.

  The storage unit 111 stores correspondence information 1000 in the present embodiment. Correspondence information 1000 is information used to identify a microphone that acquires the sound at that position from the coordinate position on the screen operated by the user. For example, it may be information in which the microphone identifier 1002 and the area name 1004 shown in FIG. Here, the area is obtained by dividing the screen operated by the user into ranges in which each microphone acquires sound. An example of the area is shown in FIG. Such correspondence information 1000 is stored in the storage unit 111 in advance.

  The communication unit 112 is a communication interface corresponding to a wired or wireless communication method. In the present embodiment, the communication unit 112 receives video from the imaging device 140 and inputs the video to the control unit 114, and also receives audio from the audio processing device 120 and inputs it to the control unit 114.

  The coordinate input unit 113 is an input device for inputting a coordinate position on the screen or a connection interface with the input device. For example, an input device such as a touch panel, a mouse, a trackball, or a joystick, or a connection interface with the above input device. In the present embodiment, the coordinate input unit 113 is a touch panel overlaid on the display unit 115, and inputs a coordinate position on the display screen of the display unit 115 in accordance with a user operation. When the user touches the screen, the touch panel reads the position and change and inputs them to the information processing apparatus 110.

  The control unit 114 has a function of controlling the movement of the information processing apparatus 110 as a whole. The control unit 114 implements each function of the information processing apparatus 110 by reading, interpreting, and executing a program describing a processing procedure in the information processing apparatus 110. The control unit 114 may execute processing in response to an input from the coordinate input unit 113. The control unit 114 may be configured by, for example, a CPU (Central Processing Unit).

  The display unit 115 is a display having a function of displaying an image or an output interface to the display. For example, examples of the display include a liquid crystal display (LCD), a plasma display (PDP), a field emission display (FED), an organic electroluminescence display (organic EL, OELD: Organic). Electroluminescence Display), a video projector, etc. are mentioned. The display unit 115 has a function of displaying an input image in accordance with control from the control unit 114.

  The audio output unit 116 is an output interface to a device having a function of outputting sound or a device having a function of outputting sound. For example, the audio output unit 116 is an output device such as a speaker, or an output interface to the speaker. The sound output unit 116 outputs the input sound according to the control from the control unit 114.

(Speech processor 120)
The audio processing device 120 mainly includes a communication unit 121, a microphone identifier assigning unit 122, an audio mixer 123, and a multiplexing unit 124.

  The communication unit 121 is a communication interface corresponding to a wired or wireless communication method. In the present embodiment, the communication unit 121 is connected to the microphone 130 and the communication unit 112 of the information processing apparatus 110, receives the voice acquired by the microphone 130, and inputs the received voice to the microphone identifier assigning unit. The communication unit 121 is an interface for inputting the voice input from the multiplexing unit 124 to the information processing apparatus 110. Further, the communication unit 121 receives control information for voice from the information processing apparatus 110 and inputs the control information to the microphone identifier assigning unit 122.

  When the control information is received from the information processing apparatus 110 with respect to the sound input from the communication unit 121, the microphone identifier assigning unit 122 assigns a microphone identifier and inputs it to the audio mixer 123. Here, the microphone identifier is a number unique to each microphone 130 assigned in advance. The microphone identifier assigning unit 122 also inputs the control information received from the communication unit 121 to the audio mixer 123 together with the audio. On the other hand, when the control information is not received from the information processing apparatus 110, the microphone identifier assigning unit 122 inputs the sound to the sound mixer without performing any processing on the sound.

  When the audio mixer 123 receives the audio with the microphone identifier from the microphone identifier assigning unit 122 and the control information, the audio mixer 123 receives the audio corresponding to the microphone identifier specified according to the control information by the microphone identifier giving unit 122. The identification is performed using the identifier, the mixing processing is performed on the identified voice according to the control information, and the audio is input to the multiplexing unit 124. If the audio mixer 123 has not received control information from the information processing apparatus 110, the audio mixer 123 does not perform any processing on the audio and inputs the audio to the multiplexing unit.

  The multiplexing unit 124 multiplexes the audio mixed in the audio mixer 123 in consideration of the sound source position and volume, and inputs the multiplexed audio to the communication unit 121. The multiplexed audio is transmitted to the network via the communication unit 121.

(Operation of the information processing system 100)
Next, the operation of the information processing system 100 according to the present embodiment will be described with reference to FIGS. 6 and 7. FIG. 6 is a flowchart for explaining the sound adjustment operation in the information processing system 100 according to the present embodiment. FIG. 7 is a flowchart showing a procedure for specifying a microphone in the information processing system 100 according to the first embodiment of the present invention. Here, FIG. 7 is a sub-flowchart of step S104 of FIG.

  First, the control unit 114 acquires the video captured by the imaging device via the communication unit 112 and displays the video on the display unit 115, and the voice acquired by the microphone 130 is transmitted to the voice processing device 120 and the communication unit 112. And output to the audio output unit 116. Then, for example, as shown in FIG. 4, when it is difficult to hear the voice of the participant 40 speaking, the user touches the finger 50 so as to point near the participant 40 on the screen. Then, the control unit 114 of the information processing apparatus 110 acquires the input coordinate position X (x, y) as a result of the touch (S102). Next, the control unit 114 of the information processing apparatus 110 specifies a microphone identifier corresponding to the input coordinate position X (x, y) (S104).

  Next, the microphone identification procedure will be described with reference to FIG. 7 which is a sub-flowchart of step S104 in FIG. First, from the input coordinate position X (x, y), the control unit 114 specifies a corresponding area (S202). In the present embodiment, it can be seen that the corresponding area is area B. This area specifying step can be realized by, for example, holding correspondence information between the coordinate position and the area in the storage unit in the information processing apparatus 110.

  And the microphone which acquires the audio | voice of the specified area is specified, and the microphone identifier is acquired (S204). In the present embodiment, the area is defined as an area for acquiring sound by each microphone. Therefore, for example, the correspondence information 1000 as shown in FIG. The microphone identifier can be specified. In the present embodiment, it can be seen that the microphone identifier of the microphone 130 that acquires the sound of the area B is 130b.

  Next, returning to FIG. 6 again, the description of the audio adjustment procedure will be continued. In step S104, when the microphone identifier to be adjusted is acquired, the control unit 114 turns on the sound adjustment function of the specified microphone (S106). Then, the control unit 114 displays an audio adjustment interface near the input coordinate position X (x, y) (S108).

  Then, for example, as shown in FIG. 2, when the user operates the audio adjustment interface 60 with the finger 50, the control unit 114 acquires the operation information (S110).

  The control unit 114 generates and outputs voice control information based on the microphone identifier specified in step S104 and the user's voice adjustment operation information acquired in step S110. The output audio control information is output to the audio processing device 120 via the communication unit 112.

  When the completion of the audio adjustment is detected (S114), the display of the audio adjustment interface is turned off and the audio adjustment is finished.

(Summary)
As described above, the information processing apparatus 100 according to the present embodiment automatically identifies the microphone 130 that has acquired the sound at a specific position on the screen on which the video is projected, using the correspondence information. The user does not need to make a correspondence as to which microphone 130 has been used to acquire the sound simply by pointing to the object whose sound is to be adjusted and performing the sound adjustment operation. Therefore, it is possible to easily adjust the voice by an intuitive operation.

(Second Embodiment)
In the first embodiment described above, the microphone that acquires the sound at the position is specified from the position on the operation screen based on preset correspondence information. However, such a system needs to set correspondence information between a microphone and an area on the screen in advance. Therefore, in the second embodiment of the present invention, the operation target microphone can be specified without the user setting correspondence information in advance. In the following description of the second embodiment, description of parts that are the same as those of the first embodiment will be omitted, and differences will be mainly described.

(Information processing system 200)
First, the information processing system 200 according to the present embodiment will be described with reference to FIG. FIG. 8 is a block diagram showing a functional configuration of an information processing system according to the second embodiment of the present invention.

  The information processing system 200 mainly includes an information processing device 210, a sound processing device 220, an imaging device 240, a plurality of sound acquisition devices 250, and a correspondence information creation device 260, which are connected by a network 300.

  In the information processing system 200, the correspondence information creation device 260 collects and creates information generated by using the voice acquisition device 250 and the batch holder 800 in order to identify the microphone that has acquired the voice specified by the user. Use correspondence information.

  Since the information processing device 210, the sound processing device 220, and the imaging device 240 are the same as those in the first embodiment, the description thereof is omitted. In addition, although the memory | storage part is not illustrated in the information processing apparatus 210, of course, the structure which has a memory | storage part may be sufficient. The information processing apparatus 210 shows that it does not have correspondence information inside. The operation at the time of volume adjustment of the information processing apparatus 210 will be described later.

(Correspondence information creation device 260)
The correspondence information creation device 260 mainly includes a communication unit 261, a control unit 262, and a storage unit 263. In this embodiment, attention is focused on the function of creating correspondence information, but the correspondence information creation device 260 can recognize the spatial arrangement of the voice acquisition device 250 and the participants by collecting the correspondence information. It is.

(Voice acquisition device 250)
The sound acquisition device 250 is a device in which a microphone for acquiring sound is installed. In the present embodiment, the voice acquisition device 250 has, for example, the desk shape shown in FIGS. 10 and 11. In FIG. 8, the voice acquisition device 250 is described as having two of the voice acquisition device 250a and the voice acquisition device 250b, but is not limited thereto. Multiple combinations can be used.

  The voice acquisition device 250 mainly includes a storage unit 251, a control unit 252, a communication unit 253, a receiver 254, a transmitter 255, and a microphone 230. Here, since there are a plurality of receivers 254 and transmitters 255, for example, the three receivers 254 are assigned the codes of the receivers 2541 to 2543 in order to identify them. Hereinafter, the transmitter will be described as the transmitter 255 when collectively described. The same applies to the receiver.

  The storage unit 251 is a storage device that can store information. For example, magnetic recording media such as HDD (Hard Disk Drive), EEPROM (Electronically Erasable and Programmable Read Only Memory), flash memory, MRAM (Magnetoresistive Random Access Memory), FeRAM (Ferroelectric Random Access Memory), PRAM (Phase change Random Access) Non-volatile memory such as “Memory”, but is not limited to the above.

  In the present embodiment, the storage unit 251 stores correspondence information 2000 as illustrated in FIG. 9, for example. FIG. 9 is an explanatory diagram illustrating an example of correspondence information in the voice acquisition device. The correspondence information 2000 stores in advance the correspondence between the voice acquisition device identifier 2002, the microphone identifier 2004, the transmitter identifier 2006, and the receiver identifier 2008 that each of the voice acquisition devices 250 has. When the identification information is received, the received identification information is stored in the reception information 2010.

  The control unit 252 controls the overall operation of the voice acquisition device 250. For example, when identification information is received by the receiver 254, the control unit 252 stores the received identification information in the reception information 2010 corresponding to the receiver identifier in the correspondence information 200 of the storage unit 251, and The correspondence information 2000 is transmitted to the correspondence information creation device 260 via the communication unit 253. The control unit 252 may be configured by a CPU, for example.

  The communication unit 253 is a communication interface for connecting the voice acquisition device 250 to a network. The communication unit 253 may be for connecting to a wired network or may be for connecting to a wireless network. The communication unit 253 transmits and receives various data according to the control of the control unit 252.

  The receiver 254 communicates with a transmitter 255 installed in another voice acquisition device 250 or a sound source identifier transmitter 801 that transmits a sound source identifier under the control of the control unit 252, and receives a transmitter identifier or a sound source identifier. Have The receiver 254 may be an infrared receiver that performs communication using infrared rays, for example. The physical installation of the receiver 254 will be described later.

  The transmitter 255 transmits a transmitter identifier toward the opposite receiver according to the control of the control unit 252. The transmitter 255 may be an infrared transmitter that performs communication using infrared rays, for example. It is assumed that the communication method of the transmitter 255 is compatible with the communication method of the receiver 254.

  The microphone 230 is a device for acquiring sound. The microphone 230 is installed in the sound acquisition device. For example, the microphone 230 may be built in the sound acquisition device 250 or may be installed on the side surface or the upper surface of the sound acquisition device 250, for example. The microphone 230 is preferably installed corresponding to the position where the participant sits in the desk-like audio acquisition device 250.

(Physical configuration)
Next, the physical configuration of the voice acquisition device 250 will be described with reference to FIGS. 10 and 11. FIG. 10 is an overview of the voice acquisition device 250a and the voice acquisition device 250b as viewed from above. FIG. 11 is a schematic view of the voice acquisition device 250a and the voice acquisition device 250b as viewed from the side.

  In the present embodiment, the voice acquisition device 250 has a desk shape. A receiver 254 and a transmitter 255 are arranged on the same side as a pair. Note that the receiver 254 and the transmitter 255 are shown in a protruding shape in the figure, but actually, in order to eliminate a gap between the desks, for example, a concave shape is formed on the side surface facing the participant of the voice acquisition device 250. It shall be provided in In the present embodiment, three receivers 254 and three transmitters 255 are provided for each voice acquisition device 250. Similar to the correspondence information (FIG. 9) indicating the correspondence on information, the physical configuration (correspondence on the configuration) between the direction of the microphone of the microphone 230 and the orientation of the desk is also associated. For example, as shown in FIG. 10, the direction of the microphone of the microphone 230a is opposite to the side surface provided with the transmitter 2551a. That is, the microphone 230a mainly acquires the voices of the participants and the like existing on the side where the transmitter 2551a is provided.

  As shown in FIG. 10, the receiver 254 and the transmitter 255 are configured such that when each side of the voice acquisition device 250 contacts, the receiver 254 of one voice acquisition device 250 and the transmitter of the other voice acquisition device 250. It arrange | positions so that 255 may oppose. For example, in FIG. 9, when the longest sides of the top of the desk are aligned, the receiver 2543a and the transmitter 2553b and the transmitter 2553a and the receiver 2543b face each other and can communicate.

  Further, as indicated by a dotted line in FIG. 10, the sound acquisition device 250 includes a microphone 230. Microphones 230 are provided corresponding to the number of receivers 254. In the present embodiment, three microphones 230 are installed for each sound acquisition device 250. The microphone 230 is associated with the receiver 254 and the transmitter 255.

  A recognition code 700 is embedded as a pattern on the top surface of the sound acquisition device 250. The recognition code 700 indicates a voice acquisition device identifier for each voice acquisition device 250. Since it is repeatedly embedded in the whole, by decoding this recognition code 700, it is possible to identify which voice acquisition device is the nearest voice acquisition device 250 at the point indicated by the user from the video showing the recognition code. It becomes possible to do. Note that the recognition code 700 according to the present embodiment will be described using an example in which a voice acquisition device identifier is shown, but is not limited to this example. For example, the recognition code 700 includes a plurality of microphones 230 as a set. An identifier for identifying a microphone set may be used. For example, as illustrated in FIG. 10, a microphone set including a pair of microphones 230 a to 230 c can be exemplified. In such a case, the voice acquisition device 250 can include a plurality of microphone sets. In this case, for example, of the surface of the voice acquisition device 250, the recognition code 700a is displayed on the peripheral portion provided with the first microphone set, and the recognition code 700b is displayed on the peripheral portion provided with the second microphone set. Is done.

  Here, the recognition code 700 is a code capable of storing the identifier of the sound acquisition device 250, and information can be read from the image. For example, the barcode may be a QR code (registered trademark), other two-dimensional code, or a one-dimensional code.

  Next, using FIG. 12 and FIG. 13, a state when the reception unit 254 of the voice acquisition device 250 receives a sound source identifier will be described. FIG. 12 is an explanatory diagram illustrating communication between the sound source identifier transmitter and the voice acquisition device. FIG. 13 is an explanatory diagram illustrating an example of an overview configuration of a batch holder.

  As shown in FIG. 12, consider a case where the participant 10 is seated on the voice acquisition device 250. A receiver 2542 is installed on the side of the voice acquisition device 250 on the side where the participant 10 is seated. At this time, the participant 10 wears the batch holder 800, and the sound source identifier transmitter 801 included in the batch holder 800 transmits the sound source identifier to the receiver 2542 of the sound acquisition device 250. As shown in FIG. 12, it is desirable that the voice acquisition device 250 be designed so that when the participant 10 is seated, the batch holder 800 and the receiver 2542 are positioned to face each other.

  Here, the configuration of the batch holder 800 will be described with reference to FIG. The batch holder 800 is for the participant 10 to wear a batch or the like for showing himself / herself, for example. For example, a sound source identifier transmitter 801 for transmitting a sound source identifier is attached to the batch holder 800. The sound source identifier is a code for identifying the sound source, and may be an identification number assigned to an individual such as an employee number.

  As described above, the information processing system 200 according to the present embodiment transmits information between the receiver 254 and the transmitter 255 provided on the side surfaces when the voice acquisition device 250 having a desk shape is combined. By transmitting and receiving the device identifier, it becomes possible to grasp which receiver is facing which transmitter. Further, if the correspondence information creation device 260 holds data such as the shape of each voice acquisition device 250 in advance, the arrangement of the voice acquisition device 250 is reproduced in the virtual space. Is possible.

  Further, for example, a sound source identifier is given to each participant 10, and the sound source identifier is transmitted to the batch holder 800 worn by the participant so that the receiver 254 at the place where the participant is seated can receive the sound source identifier. By providing the device 801, the voice acquisition device 250 and the correspondence information creation device 260 that has received the correspondence information from the voice acquisition device 250 know which participant is seated at which position of each voice acquisition device 250. Will be able to.

(Operation example)
Here, an operation example using the information processing system 200 as described above will be described. For example, an example in which four voice acquisition devices 250 are combined will be described. When the four voice acquisition devices 250 are combined, a transmitter identifier is transmitted and received between the receiver 254 and the transmitter 255 facing each other. Then, when receiving the transmitter identifier, the receiver 254 stores the received transmitter identifier in the reception information of the correspondence information in the storage unit 251 under the control of the control unit 252.

  Further, consider a case where a participant is seated on the voice acquisition device 250 combined in such a manner. Here, it is assumed that four participants are seated. When the participant is seated, the sound source identifier is transmitted from the sound source identifier transmitter of the batch holder owned by the participant to the receiver 254 of the voice acquisition device 250. The received sound source identifier is stored in the reception information of the correspondence information in the storage unit 251 under the control of the control unit 252.

  The correspondence information collected in this way is shown in FIGS. 14-17 is explanatory drawing which shows the correspondence information acquired in the audio | voice acquisition apparatus 250. FIG.

  FIG. 14 is for the voice acquisition device 250a. A voice acquisition device identifier 2102, a microphone identifier 2104, a transmitter identifier 2106, and a receiver identifier 2108 are stored in advance, and information received by the corresponding receiver 254 is stored in the reception information 2110, respectively.

  In the following, FIG. 15 shows correspondence information for 250b, FIG. 16 for 250c, and FIG. 17 for 250d. The correspondence information collected in this way is transmitted to the correspondence information creation device 260.

  When the correspondence information collected by each voice acquisition device 250 is received by the correspondence information creation device 260, the correspondence information creation device 260 is assigned to the desk shape, configuration, and participants that are held in advance. It is possible to grasp the arrangement status of the voice acquisition device 250 and the seating status of the participant using the information of the sound source identifier. That is, it is possible to grasp which sound source sound is acquired by which microphone.

  That is, the correspondence information creation device 260 uses the correspondence information shown in FIGS. 14 to 17 to indicate that the voice acquisition device is arranged as shown in FIG. 18, the position where the participant is seated, and the participation It is also possible to grasp the microphone corresponding to the position where the person is seated. FIG. 18 is an explanatory diagram showing an arrangement example of the voice acquisition devices recognized by the correspondence information creation device 260.

  In such a situation, an operation when performing voice control from the display screen of the information processing apparatus 210 while viewing the video captured by the imaging apparatus 240 will be described with reference to FIGS. 19 and 20. FIG. 19 is an explanatory diagram for explaining voice control of the information processing system according to the second embodiment of the present invention. FIG. 20 is a sub-flowchart showing a microphone identification flow for audio adjustment of the information processing system 200 according to the second embodiment of the present invention.

  The overall flow of the audio adjustment is the same as that described with reference to FIG. Since only the microphone specifying flow is different, the microphone specifying flow shown in FIG. 20 which is a sub-flowchart of step S104 of FIG. 6 will be described. As a premise, the information processing device 210 receives the correspondence information from the correspondence information creation device 260 when the correspondence information is updated, such as when the voice acquisition device 250 is arranged or when a participant is seated. It shall be acquired.

  In FIG. 19, the user touches the target whose sound is to be adjusted to point on the screen. Here, the case where the audio | voice of the participant 40 is adjusted is considered. When the participant 40 is touched to point, the pointed coordinate position is acquired. Then, the control unit 214 of the information processing apparatus 210 acquires an image at the time of inputting a coordinate position (S302). And the control part 214 searches the recognition code nearest to a coordinate position from the acquired image (S304).

  Next, the desk corresponding to the recognition code is identified from the correspondence information (S306). Specifically, when the recognition code 700 searched in step S304 is read, the voice acquisition device identifier “250d” given to each desk can be acquired. And the receiver which received the sound source identifier is specified from the correspondence information of the specified desk (S308). Referring to FIG. 17, it can be seen that the receiver 2542d is receiving the sound source identifier in the voice acquisition device 250d.

  Then, the microphone corresponding to the specified receiver 2542d is specified (S310). Here, referring to FIG. 17, it can be seen that the corresponding microphone identifier is 230k.

  In this way, it is possible to specify the microphone to be adjusted from the position on the operation screen, and reliably execute the desired microphone adjustment.

  At this time, for example, when two or more participants are sitting on the specified desk (or a microphone set including a plurality of microphones 230), the control unit 214 displays both adjustment interfaces. It may be.

(Example of effects)
As described above, according to the information processing system according to the second embodiment of the present invention, the user can automatically collect the correspondence information and specify the microphone without setting the correspondence information in advance. . Then, it becomes possible to perform audio adjustment on the identified microphone while viewing the video.

  As mentioned above, although preferred embodiment of this invention was described in detail, referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this example. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, in the first embodiment, a fixed microphone is used, but the present invention is not limited to this example. For example, a pin microphone may be used, and in this case, the position of the pin microphone is determined according to the seating position of the participant, and therefore it is necessary to associate the area information with the area information in advance.

  For example, in this embodiment, although it was set as the structure which has a display input part which laminated | stacked the touch-panel type coordinate input part on the display part, it is not restricted to this. For example, the display unit may be an interface connected to a projector that projects on a screen. In this case, an input device such as a mouse may be used.

  Further, for example, in the second embodiment, the sound acquisition device is a desk with an isosceles triangle shape, but this shape is not limited. A receiver, a transmitter, and a microphone may be paired and arranged at a position where a desk participant sits.

  Further, for example, in the second embodiment, the sound source identifier is transmitted from the batch holder worn by the participant, but is not limited to this example. For example, a sound source identifier may be assigned to each device that can be a sound source such as a projector, and a sound source identifier transmitter may be provided in the device. In that case, it is necessary to devise the arrangement of the receiver of the voice acquisition device or the communication method. For example, when the sound of a projector fan is loud, it can be used to reduce the volume.

  For example, in the second embodiment, the receiver and the transmitter use infrared rays, but the present invention is not limited to this. For example, signals may be transmitted and received using a Doppler sensor using a sound wave, a ZigBee (registered trademark) node, or the like.

  Further, for example, in the second embodiment, the recognition code is embedded only in the upper surface of the voice acquisition device, but is not limited thereto. For example, it may be embedded in the entire surface such as a side surface or a desk leg.

  For example, in the second embodiment, the information processing apparatus acquires the correspondence information from the correspondence information creation apparatus every time the correspondence information is updated. However, the present invention is not limited to this. . For example, the correspondence information may be acquired before performing the microphone identification process.

  In the second embodiment, the recognition code 700 will be described by way of an example indicating a voice acquisition device identifier, but is not limited to this example. For example, the recognition code 700 may be an identifier for identifying a microphone set including a plurality of microphones 230 as a set. For example, as illustrated in FIG. 10, a microphone set including a pair of microphones 230 a to 230 c can be exemplified. In such a case, the voice acquisition device 250 can include a plurality of microphone sets. In this case, for example, of the surface of the voice acquisition device 250, the recognition code 700a is displayed on the peripheral portion provided with the first microphone set, and the recognition code 700b is displayed on the peripheral portion provided with the second microphone set. Is done. Further, the recognition code 700 may identify each of the microphones. In this case, the voice acquisition device can be identified from the recognition code, and the microphone can be identified directly from the recognition code without being associated with the microphone.

  In this specification, the steps described in the flowcharts are executed in parallel or individually even if they are not necessarily processed in time series, as well as processes performed in time series in the described order. Including processing to be performed. Further, it goes without saying that the order can be appropriately changed even in the steps processed in time series.

DESCRIPTION OF SYMBOLS 110 Information processing apparatus 113 Coordinate input part 114 Control part 115 Display part 116 Audio | voice output part 120 Audio | voice processing apparatus 130 Microphone 140 Imaging device

Claims (7)

An audio acquisition device comprising a plurality of microphones;
An imaging device for acquiring video by shooting ;
An audio processing device for processing audio acquired by the plurality of microphones;
An information processing device that outputs the video and sound processed in the sound processing device, and outputs sound control information to the sound processing device;
With
The voice acquisition device
A receiver for receiving the sound source identifier;
On the surface of the voice acquisition device, a recognition code embedded with a voice acquisition device identifier for identifying the voice acquisition device is displayed,
The information processing apparatus includes:
A display unit for displaying images;
A coordinate input unit for inputting a coordinate position on the display screen of the display unit;
An audio adjustment interface is superimposed on the video and displayed on the display unit in the vicinity of the coordinate position input by the coordinate input unit, the audio acquisition device identifier is read from the recognition code in the video, and the read audio acquisition Based on the device identifier and the correspondence information stored in the storage unit, the microphone identifier of the microphone that acquires the sound at the position corresponding to the coordinate position in the video is identified, and the identified microphone identifier and the sound adjustment interface A control unit for outputting voice control information including voice processing information according to an input from the coordinate input unit for
I have a,
The correspondence information is generated by collecting correspondence relationships among the sound acquisition device identifier, the sound source identifier given in advance to the sound source, the receiver identifier given to the receiver, and the microphone identifier. . The voice acquisition device has a desk shape,
The receiver is installed on a side surface of the voice acquisition device,
The receiver receives the sound source identifier from a sound source identifier transmitter worn by a person;
The information processing system according to claim 1 . The voice acquisition device further includes a transmitter that transmits its own transmitter identifier,
When the transmitter is combined with another voice acquisition device, the transmitter is arranged to face the receiver of the other voice acquisition device,
The receiver receives either the sound source identifier or the transmitter identifier;
The information processing system according to claim 2 , further comprising a correspondence information creation device that acquires correspondence information of the plurality of voice acquisition devices and recognizes an arrangement of the plurality of voice acquisition devices. The voice processing device
A microphone identifier giving unit for giving a microphone identifier to the sound input from the microphone;
An audio mixer that processes the audio based on the microphone identifier assigned to the audio and the audio control information;
The a, an information processing system according to claim 1 or 2. The information processing system according to claim 1, wherein the correspondence information is information in which a correspondence relationship between the coordinate position and the microphone identifier is stored in advance. An information processing apparatus that acquires and outputs audio acquired by a plurality of microphones provided in an audio acquisition apparatus and video acquired by an imaging apparatus by shooting ,
A display unit for displaying the video;
A coordinate input unit for inputting a coordinate position on the display screen of the display unit;
An audio adjustment interface is superimposed on the video and displayed on the display unit in the vicinity of the coordinate position input by the coordinate input unit, and an audio acquisition device identifier is obtained from a recognition code displayed on the surface of the audio acquisition device in the video. The microphone identifier of the microphone that acquires the audio at the position corresponding to the coordinate position in the video is specified based on the read voice identifier and the correspondence information stored in the internal or external storage unit. A control unit that outputs audio control information including audio processing information corresponding to the input from the coordinate input unit to the specified microphone identifier and the audio adjustment interface to the audio processing device ;
The correspondence information is a correspondence relationship between the sound acquisition device identifier, a sound source identifier previously assigned to a sound source, a receiver identifier assigned to a receiver included in the sound acquisition device, and the microphone identifier. Processing equipment. A method by an information processing apparatus comprising: a display unit that displays an image; a coordinate input unit that inputs a coordinate position on a display screen of the display unit; and a control unit,
The control unit is
A voice adjustment interface is superimposed on the video and displayed on the display unit near the coordinate position input by the coordinate input unit, and the voice acquisition device identifier is read from the recognition code displayed on the surface of the voice acquisition device in the video. Identifying a microphone identifier of a microphone that acquires sound at a position corresponding to the coordinate position in the video based on the read sound acquisition device identifier and correspondence information stored in an internal or external storage unit; the audio control information including a sound processing information corresponding to the input from the coordinate input unit for the identified microphone identifier and the audio adjustment interface outputs to the audio processor,
The information processing method , wherein the correspondence information is a correspondence relationship between the sound acquisition device identifier, a sound source identifier previously assigned to a sound source, a receiver identifier assigned to a receiver included in the sound acquisition device, and the microphone identifier .

Images