JP2019046482A - Voice video tracking device - Google Patents

Abstract

To allow a smooth operation even in high noise environment or in a use state in which a touch operation cannot be easily performed.SOLUTION: A voice recognition part 2 performs voice recognition to an acoustic signal obtained by acquiring sound in the surrounding containing voice of a user by using a sound acquisition part M, and creates voice recognition result. A video acquisition part 5 acquires a video signal obtained by imaging an area corresponding to a visual field of the user by using an imaging part V. A voice creation part 4 outputs an output acoustic signal to a sound release part S. A video creation part 6 outputs an output video signal created by using the video signal to a video display part G. A function control part 3 creates a control signal for controlling functions of the video acquisition part 5, the voice recognition part 2, the voice creation part 4, and the video creation part 6 on the basis of a dial operation signal from a dial operation part D, a button operation signal from a button operation part B, and the voice recognition result.SELECTED DRAWING: Figure 2

Description

  The present invention is an audio / video input / output device that can be operated by an audio command input.

  BACKGROUND In recent years, there is a growing need for factory workers who perform difficult work and unskilled work to increase work efficiency by making full use of multimedia technology such as audio and video and information communication technology. However, carrying information communication devices such as a laptop computer and a tablet terminal separately from the tools necessary for the work poses a heavy burden on workers. In addition, although such devices mainly use keyboard operations and screen operations, workers are not always in an environment in which both hands can be freely used, and intuitive operation is often lacking. Therefore, there is a problem that it is difficult to perform the work intended by the worker, even when trying to work in cooperation with audio and video. In addition, in such an environment, there are also problems that noise is often high and communication can not be performed smoothly in many cases.

  In response to the needs as described above, a spectacles-type wearable device has been developed which displays necessary information superimposed on a real view. For example, there is Google Glass (registered trademark) described in Non-Patent Documents 1 and 2. Google Glass inputs voice commands (for example, utters "OK glass." Etc. For details, refer to Non-Patent Document 1.) and commands for gestures (for example, finger touch, body tilt state, etc.) For details, refer to Non-Patent Document 2.).

Google, Inc., "Google Glass-Help-Voice actions", [online], [April 3, 2015 search], Internet <URL: https://support.google.com/glass/answer/3079305? hl = en> Google, Inc., "Google Glass-Help-Glass gestures", [online], [April 3, 2015 search], Internet <URL: https://support.google.com/glass/answer/3064184? hl = en>

  However, conventional glasses-type wearable devices are not supposed to be used in high-noise environments. For example, in an environment such as a factory where the ambient noise is large, voice is buried in noise and command input by voice is likely to be erroneous . In addition, workers in the factory often work with thick gloves, and perform detailed operations regardless of electrostatic and piezoelectric types with touch pads that conventional glasses-type wearable devices have. Is difficult. In addition, in many cases, it is necessary in the factory to wear a helmet-like head, which physically interferes with these wearings and may not be able to wear properly. Furthermore, it is necessary to attach a fall prevention strap etc. in order to prevent a fall accident, and installation preparation takes time.

  An object of the present invention is to provide an audio / video input / output device which enables smooth operation even in a high noise environment or a use situation where touch operation is difficult.

  In order to solve the above-described problems, the audio / video input / output device of the present invention captures a plurality of sound collection units that collect surrounding sounds including the user's voice and an area corresponding to the user's field of view A photographing unit, a video display unit in which a screen is disposed at a position where the user enters the field of view, a dial operation unit that outputs a dial operation signal indicating a rotation direction and a rotation angle according to a rotation operation; A button operation unit arranged at the position of the rotation axis and outputting a button operation signal indicating a pressed state, a voice recognition unit for voice recognition of an acoustic signal acquired using the sound collection unit, and generation of a voice recognition result; And a video generation unit for outputting an output video signal generated using the video signal to the video display unit, a dial operation signal, a button operation signal, and a voice recognition result. , Image acquisition unit includes a voice recognition unit, and a function control unit for generating a control signal for controlling the function of the image generating unit.

  According to the audio / video input / output apparatus of the present invention, since voice recognition is performed using acoustic signals collected from a plurality of sound collection units, the voice command input operates stably even in a high noise environment. Moreover, since physical operation is possible by the dial and the button, the operation is easy even in a use situation where touch operation is difficult. Therefore, smooth operation is possible even in a high noise environment or a use situation where touch operation is difficult.

FIG. 1 is a diagram illustrating the functional configuration of an audio / video input / output device according to the first embodiment. FIG. 2 is a diagram illustrating the functional configuration of the audio / video input / output device according to the first embodiment. FIG. 3 is a diagram illustrating an image of using the environment setting function. FIG. 4 is a diagram illustrating an application image of the volume setting function. FIG. 5 is a diagram illustrating an application image of the image zoom function. FIG. 6 is a view exemplifying an application image of the video tracking function. FIG. 7 is a diagram illustrating the functional configuration of the audio / video input / output device according to the second embodiment. FIG. 8 is a diagram illustrating the functional configuration of the audio / video input / output device according to the second embodiment. FIG. 9 is a diagram illustrating an application image of the audio zoom function. FIG. 10 is a diagram illustrating an application image of the voice tracking function.

  Hereinafter, embodiments of the present invention will be described in detail. In the drawings, components having the same functions are denoted by the same reference numerals and redundant description will be omitted.

First Embodiment
The first embodiment of the present invention is a helmet integrated type audio / video input / output device assumed to be worn by a user on the head and used in a high noise work site such as in a factory. In the audio / video input / output apparatus of this embodiment, n (例 示 2) sound collection units M ₁ ,..., M _n , a sound emission unit S, a photographing unit V, an image display unit G, as illustrated in FIG. The dial operation unit D, the button operation unit B, and the communication recording unit C are included.

The sound collection units M ₁ ,..., M _n are microphones that collect sounds around the user when the user wears the microphones. In the example of FIG. 1, one sound collecting unit M ₁ is disposed on the mouth of the user, n-1 pieces of sound pickup unit M _2, ..., M _n showed configured to be horizontally arranged in back of the head However, any arrangement is acceptable as long as it can pick up surrounding sounds including the user's speech.

  The sound emitting unit S is a headphone disposed at a position corresponding to the user's ear when worn by the user. Headphones are generally prepared for both ears, but the sound emitting section S of this embodiment may be prepared for at least one ear, and is normally prepared for both ears as usual. It may be

  The photographing unit V is a video camera in which an angle of view is set in an area corresponding to the field of view of the user when the user wears the camera, and the area corresponding to the field of view of the user is photographed. Although the example of FIG. 1 shows the configuration of being arranged at the position of the user's forehead, the area corresponding to the field of view of the user may be arranged at any position as long as it can be photographed. However, it is desirable that the imaging unit V be disposed at a position that does not obstruct the user's visual field.

  The image display unit G is a small-sized display using a transmissive screen in which a screen is disposed at a position where the user's view comes into view when worn by the user. Since the video display unit G is of a transmissive type, it does not block the user's field of vision when there is no video output, and when there is a video output, the video is superimposed on the scene in front of the user as viewed from the user Be done. Here, since it assumes use at the work site such as in the factory, it also has the effect of protecting the user's eyes from flying objects such as dust. The image display unit G is movable and can be retracted out of the range of the field of view of the user. For example, convenience is high when confirmation with the naked eye is necessary to confirm an accurate color or the like. When being retracted, it is desirable that the storage unit be stored at a position that does not enter the angle of view of the imaging unit V.

  The dial operation unit D is disposed on the surface of the sound emission unit S opposite to the user's ear when worn by the user, and is a large-sized dial rotatable about the position of the user's ear as a central axis It is. The dial operation unit D is formed such that a plurality of irregularities are formed on the side surface of the circular dial, and for example, the user can easily operate with a fingertip even in a situation where thick gloves and the like are worn. Be done. The dial operation unit D outputs a dial operation signal d indicating the rotation direction and the rotation angle operated when the rotation operation is performed. The position of the dial operation unit D does not necessarily have to be on the surface of the sound output unit S, and may be arranged in any way as long as the user can operate it.

  The button operation unit B is a button that is disposed on the surface of the dial operation unit D at the position of the rotation axis and can be pressed in the direction of the central axis of the rotation dial. By setting the button operation unit B to be a position slightly recessed from the surface of the dial operation unit D even when the cap portion is not pressed, it is possible to prevent an erroneous operation due to an unintended button depression. The button operation unit B outputs a button operation signal b indicating a pressed state when the pressing operation is performed.

The communication recording unit C establishes a wireless or wired communication path with a remote communication party, and transmits and receives audio and video signals. Further, the audio signal and the video signal which have been transmitted and received, the audio signal acquired using the sound collection units M ₁ ,..., _Mn and the video signal acquired using the imaging unit V are recorded in a recording medium or the like.

The operation of the audio / video input / output device of the first embodiment will be described with reference to FIG. Audiovisual output device of the present embodiment, the sound collection unit M _1, ..., M _n, sound emitting section S, imaging unit V, the video display unit G, the dial operation unit D, the button operations part B, and the communication record unit C And a voice recognition unit 2, a function control unit 3, a voice generation unit 4, a video acquisition unit 5, and a video generation unit 6. As shown in FIG. 2, the communication recording unit C includes an audio output unit C1, an audio input unit C2, a video output unit C3, and a video input unit C4.

Input speech enhancement unit 1, the sound collection unit M _1, ..., acoustic signals x ₁ M _n is picked up, respectively, ..., performs target sound enhancement process on x _n, the acoustic signals x _1, ..., x _n and it outputs the audio enhancement signal a _o speech is emphasized contained. The speech enhancement signal _ao is sent to the speech recognition unit 2, speech generation unit 4, and speech output unit C1. The acoustic signals x ₁ to the sound collection unit M ₁ in the mouth of the user picked up includes a voice and environmental noise of the user, sound pickup unit M arranged in hard to reach locations the user's voice _2, ..., acoustic signals x ₂ M _n is picked up, respectively, ..., the x _n can be expected to contain only the environmental noise. Therefore, by suppressing environmental noise included in the acoustic signals x ₂ ,..., X _n in the acoustic signal x ₁ , an acoustic signal in which the user's voice is emphasized can be obtained. The target sound enhancement process is not limited to the above method, and any known method may be applied. For example, the acoustic signal enhancement technology described in the following reference 1 can be used.
[Reference 1] JP-A-2013-179388

Speech recognition unit 2 performs speech recognition on the speech enhancement signals a _o, and outputs the voice recognition results op. Any known method may be applied to the speech recognition process. The speech recognition result op is sent to the function control unit 3. The speech recognition unit 2 does not have to execute the speech recognition process itself, and transmits the speech enhancement signal _ao to the speech recognition device installed remotely, and the speech recognition result returned from the speech recognition device May be output as the speech recognition result op. At this time, communication with the voice recognition device may be performed using the communication recording unit C.

Based on the dial operation signal d from the dial operation unit D, the button operation signal b from the button operation unit B, and the speech recognition result op from the speech recognition unit 2, the function control unit 3 functions as an audio / video input / output device. Control signals c ₁ ,..., C ₅ are generated. The content of the control signal is determined by the content selected by the user through dial operation, button operation, and voice command input according to the display on the video display unit G. Although an operation example for individual functions will be described later, how to combine the dial operation, the button operation, and the voice command input to configure the operation interface is optional. For example, as a basic operation, it is conceivable to call a function by voice command input, select an option by dial operation, and confirm the selection by button operation. Moreover, it is also possible to perform from specification of an option to determination collectively by voice command input. The control signal is individually generated in accordance with the component to be controlled. For example, the control signal c ₁ is a signal for controlling for the input speech enhancement unit 1. The control signal c ₂ is a signal that controls the sound generation unit 4. Control signal c ₃ is a signal for controlling for the video acquisition unit 5. Control signal c ₄ is a signal for controlling for the image generator 6. Control signal c ₅ is a signal for controlling for the speech recognition unit 2.

The voice output unit C1 transmits the voice emphasis signal _ao output from the input voice emphasis unit 1 to the remote communication counterpart. Alternatively, the voice emphasis signal _ao is stored in a recording medium (not shown).

The voice input unit C2 receives a remote voice signal _ai from a remote communication party. The received remote voice signal _ai is sent to the voice generation unit 4. The remote voice signal _ai is, for example, a voice for instructing the user of the work to be performed by the remote communication party.

Sound generation unit 4, speech enhancement signal outputs of the input speech enhancement unit 1 a _o and the output to the remote audio signal a _i if there is a remote audio signal a _i and the output sound signal a _s using a speech input unit C2 , and outputs the output sound signal a _s to the sound emitting section S. The voice emphasis signal _ao is used to feed back the voice of the user, for example, in a high noise environment and when it is difficult for the user's own voice to be heard. Also, the voice recorded in advance may be stored in the communication recording unit C or the like, and may be used as the remote voice signal _ai by reproducing the voice.

The video acquisition unit 5 acquires a video signal _vo captured using the imaging unit V. The acquired video signal _vo is sent to the video output unit C3.

The video output unit C3 transmits the video signal _vo output from the video acquisition unit 5 to the remote communication counterpart. Alternatively, the video signal _vo is stored in a recording medium (not shown).

The video input unit C4 receives a remote video signal v _i from a remote communication party. The received remote video signal v _i is sent to the video generation unit 6. The remote video signal v _i is, for example, reference information or the like required for the operation to be performed by the user.

Image generating unit 6, generates a video signal v _o output from the image capturing section 5, an output video signal v _s using the remote video signal v _i If the remote video signal v _i output from the image input unit C4 The output video signal v _s is output to the video display unit G.

FIG. 3 shows a usage image when performing environment setting in the audio / video input / output device of this embodiment. First, the user inputs "environment" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the environment setting function to the video generation unit 6 based on the voice recognition result op of “environment”. The video generation unit 6 displays the current environment setting on the transparent screen of the video display unit G. In the example of FIG. 3, the current environment setting is “construction site”, and “tunnel”, “server room”, etc. are displayed as other setting candidates. These options and the operation parameters associated with them are set in advance. When the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and the control signal c ₄ for changing the environment setting candidate is sent to the video generation unit 6. And presses the button operation unit B after selecting the desired configuration candidate by dial operation, button operation signal b is input to the function control unit 3, the control signals c ₁ and c ₅ for confirming the change of configuration It is sent to the input speech emphasizing unit 1 and the speech recognition unit 2. The input speech emphasizing unit 1 and the speech recognition unit 2 change the operation parameters in accordance with the control signals c ₁ and c ₅ . As the target sound emphasizing process and the speech recognition process generally have different optimum operating parameters depending on the environment, it is expected that more accurate processing results can be obtained by setting the usage environment correctly.

FIG. 4 shows a use image when performing volume setting in the audio / video input / output device of this embodiment. First, the user inputs "volume" by voice command input. The function control unit 3 sends a control signal c ₄ for calling a volume setting function to the video generation unit 6 based on the voice recognition result op of “volume”. The image generation unit 6 displays the current volume setting on the transparent screen of the image display unit G. In the example of FIG. 4, the current volume setting is “50”. When the user rotates the dial operation unit D, dialing signal d is input to the function control unit 3, the control signal c ₂ for changing the volume setting to the sound generating unit 4, for changing the volume display The control signal c ₄ is sent to the video generation unit 6. Sound generation unit 4 to lower the volume of the output sound signal a _s to be output to the sound emitting unit S according to the control signal c _2. Image generation unit 6 to lower the volume display in accordance with the control signal c _4.

FIG. 5 shows a usage image when performing video zoom in the audio / video input / output device of this embodiment. First, the user inputs "image zoom" by inputting a voice command. The function control unit 3 sends a control signal c ₄ for calling the video zoom function to the video generation unit 6 based on the voice recognition result op of “video zoom”. The video generation unit 6 displays the video currently captured by the imaging unit V on the transmissive screen of the video display unit G. When the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and the control signal c ₃ for changing the magnification of the video zoom is sent to the video acquisition unit 5. The video acquisition unit 5 changes the magnification of the imaging unit V to acquire a video signal _vo . There is a problem that a camera alone attached to an overhead person or the like does not know the subject accurately when photographing an image with hands-free operation, and it is difficult to perform an operation such as directly touching the photographing unit V. If it is possible to perform video zoom by the above-described operation, it is possible to smoothly perform video range confirmation and video zoom operation.

FIG. 6 shows a usage image when performing video tracking in the audio / video input / output device of this embodiment. First, the user inputs "image tracking" by voice command input. The function control unit 3 sends a control signal c ₄ for calling a video tracking function to the video generation unit 6 based on the voice recognition result op of “video tracking”. The video generation unit 6 displays the tracking target candidate in the video in the video currently captured by the imaging unit V on the transparent screen of the video display unit G while highlighting it. In the example of FIG. 6, three objects are displayed as tracking target candidates, and an object located at the upper left of the transparent screen is highlighted as a tracking target candidate. Here, when the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and the control signal c ₄ for switching the tracking target candidate is sent to the video generation unit 6. When the user selects a desired tracking target candidate by dial operation and presses the button operation unit B in that state, the button operation signal b is input to the function control unit 3 and the control signal c ₃ for determining the tracking target Are sent to the image acquisition unit 5. Video acquisition unit 5 starts video tracking of the tracking target determined in accordance with the control signal c _3. Thereafter, as long as the tracking target is present at the angle of view of the imaging unit V, the video signal _vo centered on the tracking target is captured. By combining video tracking and video zooming, it is possible to zoom and track any object within the field of view. In this case, the imaging unit V needs to be compatible with the pan-tilt-zoom function. It is also possible to select a tracking target by voice command input in a situation where it is difficult for the user to dial. The user may switch the tracking target candidate by inputting a voice command "right", "left" or the like, or may select the tracking target candidate by directly inputting a voice command on the screen. By selecting the tracking target by voice command input, a completely hands-free operation becomes possible.

Second Embodiment
The audio / video input / output device of the second embodiment has n (≧ 2) sound collection units M ₁ ,..., M _n , a sound emission unit S, a photographing unit V, an image display unit as illustrated in FIG. G, dial operation unit D, the button operations part B, and includes a communication recording portion C as in the first embodiment, the front sound pickup unit M _{n + 1} of the further mn number (m ≧ 4), ..., a M _m Including. Further, as illustrated in FIG. 8, the audio / video input / output device of the present embodiment further includes a target sound emphasis unit 7 in addition to each component of the audio / video input / output device of the first embodiment.

Front sound pickup unit _{M n + 1, ..., M} m is a microphone for picking up sound by the user comes from a direction corresponding to the field of view of the user when worn. In Figure 7, the front sound pickup unit M _{n + 1,} ..., an example is shown in which M _m is horizontally arranged in photographing the vicinity V of the forehead, sound arriving from a direction corresponding to the field of view of the user Any arrangement is acceptable as long as it can pick up sound.

Target sound enhancement unit 7, the front sound pickup unit M _{n + 1,} ..., front audio signal x _{n + 1} M _m is picked up, respectively, ..., it performs target sound enhancement process on x _m, specific sound The target sound emphasizing signal a _o2 is output. The target sound emphasis signal _ao2 is sent to the voice generation unit 4 and the voice output unit C1. The identification of the sound to be emphasized is performed by the user using voice command input, dial operation and button operation. Specific specific operations will be described later. The target sound emphasis processing may apply any method known in the art, and for example, the acoustic signal emphasis technology described in the above-mentioned reference 1 can be used.

FIG. 9 shows a usage image when performing audio zoom in the audio / video input / output device of this embodiment. The audio zoom is a function to intensively collect sounds coming from a specific sound source. First, the user inputs "voice zoom" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the voice zoom function to the video generation unit 6 based on the voice recognition result op of “voice zoom”. The image generation unit 6 displays the current setting of the audio zoom (center position and magnification) on the transparent screen of the image display unit G. In the example of FIG. 9, the center of the thick circle indicates the zoom position, and the radius of the circle indicates the zoom magnification. The user changes the audio zoom setting by dial operation and button operation. First, when the dial operation unit D is rotated, the horizontal coordinate of the zoom position moves on the transparent screen. When the button operation unit B is pressed at a desired position, the abscissa is determined. Next, when the dial operation unit D is rotated, the vertical coordinate of the zoom position moves on the transparent screen. When the button operation unit B is pressed at a desired position, the ordinate is determined. Thereafter, when the dial operation unit D is rotated, the zoom magnification changes. Finally, when the button operation unit B is pressed, the control signal control signal c ₁ for determining the change of the audio zoom setting is sent to the target sound emphasis unit 7. The target sound emphasizing unit 7 outputs a target sound emphasizing signal _{ao2 in} which a sound coming from a specified direction is emphasized at a specified magnification. The audio zoom is considered to be used for recording video and audio from the viewpoint of the wearer at a construction site or a disaster site. In particular, when selectively recognizing the sound of a person or object at a long distance under noise, the sound zoom setting can be performed in conjunction with the transmissive screen, so that intuitive and efficient operation can be performed. Those instructions as described above, the wearer's eye position and a forward sound pickup unit M _{n + 1,} ..., can be realized because it is a helmet integrated device that the position relation of M _m is close and constant It is.

FIG. 10 shows a usage image when performing audio tracking in the audio / video input / output device of this embodiment. First, the user inputs "voice tracking" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the voice tracking function to the video generation unit 6 based on the voice recognition result op of “voice tracking”. The video generation unit 6 displays the tracking target candidate in the video in the video currently captured by the imaging unit V on the transparent screen of the video display unit G while highlighting it. In the example of FIG. 10, three objects are displayed as tracking target candidates, and an object located at the upper left of the transmissive screen is highlighted as a tracking target candidate. When the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and the control signal c ₄ for switching the tracking target candidate is sent to the video generation unit 6. When the user selects a desired tracking target candidate by dial operation and presses button operation unit B in that state, button operation signal b is input to function control unit 3 and control signal c ₁ for determining the tracking target. Are sent to the target sound emphasis unit 7. Target sound enhancement unit 7 starts the tracked audio track determined in accordance with the control signal c _1. Thereafter, as long as the tracking target is present at the angle of view of the imaging unit V, the target sound emphasis signal _ao2 in which the sound coming from the direction of the tracking target is emphasized is output. It is also possible to specify the magnification of speech enhancement by combining speech tracking and speech zoom. It is also possible to select a tracking target by voice command input in a situation where it is difficult for the user to dial. The user may switch the tracking target candidate by inputting a voice command "right", "left" or the like, or may select the tracking target candidate by directly inputting a voice command on the screen. By selecting the tracking target by voice command input, a completely hands-free operation becomes possible.

Third Embodiment
In the first embodiment and the second embodiment, the voice generation unit 4 may be configured to have an external noise suppression function. In this case, the sound generation unit 4 acquires environmental noise around the sound emission unit S using an existing sound collection unit, a front sound collection unit, or a dedicated sound collection unit, and generates a signal for suppressing the environmental noise. Then, environmental noise is suppressed by adding to the output acoustic signal a _s . With such a configuration, when utilized in noisy, there is an effect that the output acoustic signal a _s outputted from the sound emitting portion S is easily heard more for the user. As the external noise suppression processing, any known method may be applied, and for example, the noise suppression technology described in the following reference 2 can be used.
[Reference 2] Japanese Patent Application Laid-Open No. 7-303135

  The points of the audio / video input / output device of the present invention are as follows. The first point is that a plurality of microphones for emphasizing target sound are installed in the integrated helmet so that the user's speech and surrounding sounds can be easily collected. The second point is an intuitive interface for controlling the camera and multiple microphones equipped with a transparent screen, voice command input, dial operation, button operation functions, and intuitive and efficient control of video and audio The point is that it is possible to

  Since the audio / video input / output device of the present invention has the target sound emphasis function, the audio of the wearer can be collected clearly even in a high noise environment. As a result, voice command input can be stably used as means for conveying the wearer's intention to the device. In addition, by adopting a helmet integrated type device equipped with a dial and a transparent screen, it becomes possible to intuitively operate parameters such as direction, emphasis ratio and magnification for photographing and sound collection by a microphone or a camera. Furthermore, the integrated type allows the wearer to perform work such as receiving instructions remotely while sharing moving images of the site with the other party while keeping both hands free. In particular, in cases where it is necessary to work while communicating with a remote instructor at a dangerous place such as a high noise place or a construction site, or checking electronic data, the audio / video input / output device of the present invention It is possible to work safely, accurately and efficiently by using.

  The present invention is not limited to the above-described embodiment, and it is needless to say that changes can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment are not only executed chronologically according to the order described, but may be executed in parallel or individually depending on the processing capability of the apparatus executing the process or the necessity.

[Program, recording medium]
When various processing functions in each device described in the above embodiments are implemented by a computer, the processing content of the function that each device should have is described by a program. By executing this program on a computer, various processing functions in each of the above-described devices are realized on the computer.

  The program describing the processing content can be recorded in a computer readable recording medium. As the computer readable recording medium, any medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, etc. may be used.

  Further, the distribution of this program is carried out, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD, a CD-ROM, etc. in which the program is recorded. Furthermore, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, at the time of execution of the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of this program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer Each time, processing according to the received program may be executed sequentially. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to the computer It may be Note that the program in the present embodiment includes information provided for processing by a computer that conforms to the program (such as data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, in this embodiment, although the present apparatus is configured by executing a predetermined program on a computer, at least a part of the processing contents may be realized as hardware.

M sound pickup unit S sound emission unit V photographing unit G video display unit C communication recording unit D dial operation unit B button operation unit 1 input speech emphasizing unit 2 speech recognition unit 3 function control unit 4 speech generation unit 5 image acquisition unit 6 image Generation unit 7 Target sound emphasis unit C1 Audio output unit C2 Audio input unit C3 Video output unit C4 Video input unit

Claims (2)

A plurality of sound pickup units for picking up ambient sounds including the user's voice and environmental noise;
An imaging unit for imaging an area corresponding to the field of view of the user;
A video display unit in which a screen is disposed at a position where the user's view comes into view;
A dial operation unit that outputs a dial operation signal indicating a rotation direction and a rotation angle according to the rotation operation;
A button operation unit disposed at the position of the rotation axis on the surface of the dial operation unit and outputting a button operation signal indicating a pressed state;
A voice recognition unit that generates voice recognition results by performing voice recognition on an acoustic signal acquired using the above-described sound collection unit;
A video acquisition unit that acquires a video signal using the imaging unit;
A video generation unit that outputs an output video signal generated using the video signal to the video display unit;
A function control unit that generates control signals for controlling the functions of the video acquisition unit, the speech recognition unit, and the video generation unit based on the dial operation signal, the button operation signal, and the speech recognition result; Including
The multiple sound pickup units are
A first microphone placed at the mouth of the user to pick up the voice of the user and the environmental noise, and a position where the voice of the user does not easily reach to pick up only the environmental noise And a second microphone disposed;
By alternately operating the dial operation unit and the button operation unit, it is possible to change the coordinates and the range to be subjected to audio zoom in the image displayed on the image display unit, and further
If the voice recognition result is to call a voice tracking function, the function control unit may:
Highlighting a plurality of tracking target candidates included in the video signal on the display unit;
The selection of the tracking target candidate is switched based on the dial operation signal based on the user's operation and the button operation signal or the voice command issued by the user, and the selected tracking target candidate is set as the tracking target.
As long as the tracking target is present at the angle of view of the imaging unit, the audio / video input / output device continues to make sound coming from the direction selected as the tracking target as a target of audio zoom. An audio / video input / output device according to claim 1, wherein
The second microphone is disposed in a direction opposite to the direction corresponding to the field of view of the user with the user at the center,
The plurality of sound pickup units further include a third microphone disposed in the vicinity of the photographing unit to pick up sound coming from a direction corresponding to the field of view of the user;
Audio / video input / output device.

Images