JP6628853B2 - Audio-video tracking device - Google Patents

Description

  The present invention is an audio / video input / output device that can be operated by inputting an audio command.

  2. Description of the Related Art In recent years, there has been an increasing need for factory workers and the like who perform difficult or unskilled work to increase work efficiency by making full use of multimedia technology such as audio and video and information communication technology. However, carrying information communication devices such as a notebook computer and a tablet terminal separately from tools required for the work puts a heavy burden on workers. In such devices, keyboard operation and screen operation are mainly performed, but an operator is not always in an environment where both hands can be freely used, and in many cases, intuitive operability is lacking. For this reason, there is a problem that it is difficult to perform the work intended by the worker even when trying to work in cooperation with audio and video. In such an environment, there is also a problem that noise is often loud and communication cannot be performed smoothly in many cases.

  Eyeglass-type wearable devices have been developed that display necessary information in a manner superimposed on a real field of view in response to the needs described above. For example, there is Google Glass (registered trademark) described in Non-Patent Documents 1 and 2. Google Glass uses command input by voice (for example, uttering “OK glass.” For details, refer to Non-Patent Document 1) and command input using gestures (for example, touching a finger or tilting the main body) For details, see Non-Patent Document 2.).

Google, Inc., “Google Glass-Help-Voice actions”, [online], [Search April 3, 2015], Internet <URL: https://support.google.com/glass/answer/3079305? hl = en> Google, Inc., “Google Glass-Help-Glass gestures”, [online], [Search April 3, 2015], Internet <URL: https://support.google.com/glass/answer/3064184? hl = en>

  However, conventional eyeglass-type wearable devices are not intended for use in a high-noise environment. For example, in a loud environment such as in a factory, voice is buried in noise, and command input by voice is apt to be erroneous. . Workers in factories often wear thick gloves, and touchpads that are equipped with conventional eyeglass-type wearable devices require detailed operations regardless of whether they are electrostatic or piezoelectric. Is difficult. In addition, in a factory, there is often a need for a headwear such as a helmet, and there is a case where physical interference with these wears and a proper wearing method cannot be performed. Further, it is necessary to wear a fall prevention strap or the like in order to prevent a fall accident, and it takes time to prepare for wearing.

  An object of the present invention is to provide an audio-video input / output device that enables smooth operation even in a high-noise environment or a use situation where touch operation is difficult.

  In order to solve the above-described problems, an audio-video input / output device according to the present invention captures a plurality of sound collection units that collect surrounding sounds including a user's voice and an area corresponding to the user's field of view. A photographing unit, a video display unit in which a screen is arranged at a position within a user's field of view, a dial operation unit that outputs a dial operation signal indicating a rotation direction and a rotation angle according to a rotation operation, and a surface of the dial operation unit A button operation unit that is arranged at the position of the rotation axis and outputs a button operation signal indicating a pressed state, a voice recognition unit that generates a voice recognition result by voice-recognizing an acoustic signal acquired using the sound collection unit, A video acquisition unit that acquires a video signal using the video signal unit, a video generation unit that outputs an output video signal generated using the video signal to the video display unit, and a dial operation signal, a button operation signal, and a voice recognition result. hand, Image acquisition unit includes a voice recognition unit, and a function control unit for generating a control signal for controlling the function of the image generating unit.

  Since the audio / video input / output device of the present invention performs audio recognition using acoustic signals collected from a plurality of sound collection units, voice command input stably operates even in a high noise environment. In addition, since physical operation using dials and buttons is possible, operation is easy even in usage situations where touch operation is difficult. Therefore, smooth operation is possible even in a high-noise environment or a use situation where touch operation is difficult.

FIG. 1 is a diagram illustrating a functional configuration of the audio-video input / output device according to the first embodiment. FIG. 2 is a diagram illustrating a functional configuration of the audio-video input / output device according to the first embodiment. FIG. 3 is a diagram illustrating a usage image of the environment setting function. FIG. 4 is a diagram illustrating a usage image of the volume setting function. FIG. 5 is a diagram illustrating a usage image of the video zoom function. FIG. 6 is a diagram illustrating a usage image of the video tracking function. FIG. 7 is a diagram illustrating a functional configuration of the audio-video input / output device according to the second embodiment. FIG. 8 is a diagram illustrating a functional configuration of the audio / video input / output device according to the second embodiment. FIG. 9 is a diagram illustrating a usage image of the audio zoom function. FIG. 10 is a diagram illustrating a usage image of the voice tracking function.

  Hereinafter, embodiments of the present invention will be described in detail. In the drawings, components having the same function are denoted by the same reference numerals, and redundant description is omitted.

[First Embodiment]
The first embodiment of the present invention is a helmet-integrated audio / video input / output device that is assumed to be worn on the head and used by a user in a high noise work site such as in a factory. As illustrated in FIG. 1, the audio-video input / output device according to the present embodiment includes n (≧ 2) sound collecting units M ₁ ,..., M _n , a sound emitting unit S, a photographing unit V, a video display unit G, It includes a dial operation unit D, a button operation unit B, and a communication recording unit C.

The sound pickup units M ₁ ,..., M _n are microphones that pick up sounds around the user when worn by the user. In the example of FIG. 1, one sound collecting unit M ₁ is disposed on the mouth of the user, n-1 pieces of sound pickup unit M _2, ..., M _n showed configured to be horizontally arranged in back of the head However, any arrangement may be used as long as surrounding sounds including the utterance of the user can be collected.

  The sound emitting unit S is a headphone arranged at a position corresponding to the user's ear when worn by the user. Headphones are generally prepared for both ears, but the sound emitting section S of this embodiment only needs to be prepared for at least one ear, and is normally prepared for both ears. May be.

  The imaging unit V is a video camera that sets an angle of view in an area corresponding to the user's field of view when worn by the user, and captures an area corresponding to the user's field of view. In the example of FIG. 1, the configuration in which the image is arranged at the position of the user's forehead is shown. However, the image may be arranged at any position as long as an area corresponding to the user's visual field can be photographed. However, it is desirable that the photographing unit V is arranged at a position that does not obstruct the visual field of the user.

  The image display unit G is a small-sized display using a transmissive screen in which a screen is arranged at a position within a user's visual field when worn by the user. Since the image display unit G is a transmission type, it does not obstruct the user's field of view when there is no image output, and when there is an image output, the image is superimposed on the scene in front of the user and visually recognized. Is done. Here, since it is assumed to be used at a work site such as in a factory, there is also an effect of protecting a user's eyes from flying substances such as dust. The image display unit G is a movable type that can be retracted outside the range of the user's field of view. For example, when it is necessary to confirm with the naked eye such as confirming an accurate color, the convenience is high. It is desirable that when retracted, it be stored in a position that does not enter the angle of view of the photographing unit V.

  The dial operating unit D is arranged on a surface of the sound emitting unit S that is opposite to the user's ear when worn by the user, and is rotatable about the position of the user's ear as a central axis. It is. The dial operation part D has a plurality of irregularities formed on the side surface of the circular dial, and is formed so that, for example, even when the user is wearing thick gloves, the operation can be easily performed with a fingertip. Is done. The dial operation unit D outputs a dial operation signal d indicating a rotation direction and a rotation angle operated when the rotation operation is performed. The position of the dial operation unit D does not necessarily have to be on the surface of the sound emitting unit S, and may be arranged in any manner as long as the position can be operated by the user.

  The button operation unit B is a button that is disposed at the position of the rotation axis on the surface of the dial operation unit D and that can be pressed in the direction of the center axis of the rotation dial. By setting the button operation unit B so that it is slightly depressed from the surface of the dial operation unit D even when the cap is not pressed, it is possible to prevent malfunction due to unintended button press. The button operation unit B outputs a button operation signal b indicating a pressed state when a pressing operation is performed.

The communication recording unit C establishes a wireless or wired communication path with a remote communication partner, and transmits and receives an audio signal and a video signal. In addition, an audio signal and a video signal transmitted and received, an audio signal acquired by using the sound pickup units M ₁ ,..., M _n and a video signal acquired by using the imaging unit V are recorded on a recording medium or the like.

With reference to FIG. 2, the operation of the audiovisual input / output device of the first embodiment will be described. Audiovisual output device of the present embodiment, the sound collection unit M _1, ..., M _n, sound emitting section S, imaging unit V, the video display unit G, the dial operation unit D, the button operations part B, and the communication record unit C In addition to the above, an input voice emphasis unit 1, a voice recognition unit 2, a function control unit 3, a voice generation unit 4, a video acquisition unit 5, and a video generation unit 6 are included. As shown in FIG. 2, the communication recording unit C includes an audio output unit C1, an audio input unit C2, a video output unit C3, and a video input unit C4.

Input speech enhancement unit 1, the sound collection unit M _1, ..., acoustic signals x ₁ M _n is picked up, respectively, ..., performs target sound enhancement process on x _n, the acoustic signals x _1, ..., x _n and it outputs the audio enhancement signal a _o speech is emphasized contained. The voice emphasis signal _ao is sent to the voice recognition unit 2, the voice generation unit 4, and the voice output unit C1. The acoustic signals x ₁ to the sound collection unit M ₁ in the mouth of the user picked up includes a voice and environmental noise of the user, sound pickup unit M arranged in hard to reach locations the user's voice _2, ..., acoustic signals x ₂ M _n is picked up, respectively, ..., the x _n can be expected to contain only the environmental noise. Therefore, by suppressing the environmental noise included in the audio signals x ₂ ,..., X _n in the audio signal x ₁ , an audio signal emphasizing the user's voice can be obtained. The target sound emphasis processing is not limited to the above method, and any known method may be applied. For example, an acoustic signal enhancement technique described in Reference Document 1 below can be used.
[Reference Document 1] JP 2013-179388 A

Speech recognition unit 2 performs speech recognition on the speech enhancement signals a _o, and outputs the voice recognition results op. For the voice recognition processing, any known method may be applied. The voice recognition result op is sent to the function control unit 3. The voice recognition unit 2 does not need to execute the voice recognition process itself, but transmits the voice emphasis signal _ao to a remotely installed voice recognition device, and returns a voice recognition result returned from the voice recognition device. May be output as the voice recognition result op. At this time, communication with the voice recognition device may be performed using the communication recording unit C.

Based on the dial operation signal d from the dial operation unit D, the button operation signal b from the button operation unit B, and the voice recognition result op from the voice recognition unit 2, the function control unit 3 control signals c ₁ for controlling, ..., to produce a c _5. The content of the control signal is determined by the content selected by the user through dial operation, button operation, and voice command input according to the display on the video display unit G. Although an operation example for an individual function will be described later, how to combine the dial operation, the button operation, and the voice command input to configure the operation interface is arbitrary. For example, as a basic operation, a flow in which a function is called by inputting a voice command, an option is selected by a dial operation, and the selection is confirmed by a button operation may be considered. Further, it is also possible to collectively perform the process from designation of an option to determination by inputting a voice command. The control signals are individually generated according to the components to be controlled. For example, the control signal c ₁ is a signal for controlling the input voice emphasizing unit 1. The control signal c ₂ is a signal for controlling the sound generator 4. The control signal c ₃ is a signal for controlling the video acquisition unit 5. The control signal c ₄ is a signal for controlling the video generator 6. Control signal c ₅ is a signal for controlling for the speech recognition unit 2.

The audio output unit C1 transmits the audio enhancement signal _ao output from the input audio enhancement unit 1 to a remote communication partner. Alternatively, the voice enhancement signal _ao is stored in a recording medium (not shown).

The voice input unit C2 receives a remote voice signal _ai from a remote communication partner. The received remote audio signal a _i is sent to the audio generation unit 4. The remote voice signal _ai is, for example, a voice that instructs a user to perform a task to be performed by a remote communication partner.

Sound generation unit 4, speech enhancement signal outputs of the input speech enhancement unit 1 a _o and the output to the remote audio signal a _i if there is a remote audio signal a _i and the output sound signal a _s using a speech input unit C2 , and outputs the output sound signal a _s to the sound emitting section S. The voice emphasis signal _ao is used to feed back the voice of the user, for example, when the voice is used in a high-noise environment and it is difficult to hear the user's own voice. Alternatively, the voice recorded in advance may be stored in the communication recording unit C or the like, and the voice may be reproduced and used as the remote voice signal _ai .

The video acquisition unit 5 acquires a video signal v _o captured using the imaging unit V. The obtained video signal v _o is sent to the video output unit C3.

The video output unit C3 transmits the video signal _vo output from the video acquisition unit 5 to a remote communication partner. Alternatively, the video signal _vo is stored in a recording medium (not shown).

Image input unit C4 receives remote video signals v _i from a remote communication partner. Received remote video signal v _i is sent to the image generation unit 6. Remote video signal v _i is, for example, reference information required in the work to be performed by the user.

Image generating unit 6, generates a video signal v _o output from the image capturing section 5, an output video signal v _s using the remote video signal v _i If the remote video signal v _i output from the image input unit C4 and outputs the output video signal v _s to the video display unit G.

FIG. 3 is an image used when setting the environment in the audio-video input / output device of the present embodiment. First, the user inputs "environment" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the environment setting function to the video generation unit 6 based on the voice recognition result op “environment”. The image generation unit 6 displays the current environment setting on the transmission screen of the image display unit G. In the example of FIG. 3, the current environment setting is “construction site”, and “tunnel”, “server room”, and the like are displayed as other setting candidates. These options and the operation parameters associated with them are set in advance. When the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and the control signal c ₄ for changing the environment setting candidate is sent to the video generation unit 6. And presses the button operation unit B after selecting the desired configuration candidate by dial operation, button operation signal b is input to the function control unit 3, the control signals c ₁ and c ₅ for confirming the change of configuration It is sent to the input voice emphasis unit 1 and the voice recognition unit 2. Input speech enhancement unit 1 and the speech recognition unit 2 changes the operating parameter in accordance with the control signals c ₁ and c _5. In the target sound emphasizing process and the voice recognition process, it is general that optimal operation parameters differ depending on the environment. Therefore, it is expected that a more accurate processing result can be obtained by correctly setting the use environment.

FIG. 4 is an image of use when setting the volume in the audio / video input / output device of the present embodiment. First, the user inputs "volume" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the volume setting function to the video generation unit 6 based on the voice recognition result op of “volume”. The image generation unit 6 displays the current volume setting on the transmission screen of the image display unit G. In the example of FIG. 4, the current volume setting is “50”. When the user rotates the dial operation unit D, a dial operation signal d is input to the function control unit 3, and a control signal c ₂ for changing the volume setting is sent to the voice generation unit 4 for changing the volume display. The control signal c ₄ is sent to the video generator 6. Sound generation unit 4 to lower the volume of the output sound signal a _s to be output to the sound emitting unit S according to the control signal c _2. Image generation unit 6 to lower the volume display in accordance with the control signal c _4.

FIG. 5 is an image used when performing video zoom in the audio / video input / output device of the present embodiment. First, the user inputs "video zoom" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the video zoom function to the video generation unit 6 based on the audio recognition result op “video zoom”. The video generation unit 6 displays the video currently captured by the capturing unit V on the transmission screen of the video display unit G. When the user rotates the dial operation unit D, a dial operation signal d is input to the function control unit 3, and a control signal c ₃ for changing the magnification of the image zoom is sent to the image acquisition unit 5. The video acquisition unit 5 acquires the video signal _vo by changing the magnification of the imaging unit V. There is a problem that when photographing an image in a hands-free manner, the subject alone cannot be accurately recognized by a photographer with a single camera mounted overhead or the like, and it is difficult to perform an operation of directly touching the photographing unit V. If the image zoom can be performed by the above-described operation, it is possible to smoothly check the image range and smoothly perform the image zoom operation.

FIG. 6 is an image used when performing video tracking in the audio / video input / output device of the present embodiment. First, the user inputs "video tracking" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the video tracking function to the video generation unit 6 based on the voice recognition result op of “video tracking”. The image generation unit 6 displays on the transmission screen of the image display unit G while highlighting the tracking target candidate in the image in the image currently captured by the imaging unit V. In the example of FIG. 6, three objects are displayed as tracking target candidates, and an object located at the upper left of the transmissive screen is highlighted as a tracking target candidate. Here, when the user rotates the dial operation unit D, a dial operation signal d is input to the function control unit 3, and a control signal c ₄ for switching the tracking target candidate is sent to the video generation unit 6. When the user selects a desired tracking target candidate by a dial operation and presses the button operation unit B in that state, a button operation signal b is input to the function control unit 3 and a control signal c ₃ for determining the tracking target is set. Is sent to the video acquisition unit 5. Video acquisition unit 5 starts video tracking of the tracking target determined in accordance with the control signal c _3. Thereafter, as long as the tracking target exists at the angle of view of the imaging unit V, the video signal _vo centering on the tracking target is captured. By combining the video tracking and the video zoom, it is possible to track any object in the field of view while zooming. In this case, the photographing unit V needs to support the pan / tilt / zoom functions. Assuming a situation where it is difficult for the user to perform a dial operation, a tracking target can be selected by voice command input. A method is conceivable in which the user switches the tracking target candidate by inputting a voice command such as “right” or “left”, or selects the tracking target candidate by inputting a voice command directly on the screen. Selecting a tracking target by inputting a voice command enables completely hands-free operation.

[Second embodiment]
As illustrated in FIG. 7, the audio-video input / output device of the second embodiment includes n (≧ 2) sound collecting units M ₁ ,..., M _n , a sound emitting unit S, a photographing unit V, and a video display unit. G, dial operation unit D, the button operations part B, and includes a communication recording portion C as in the first embodiment, the front sound pickup unit M _{n + 1} of the further mn number (m ≧ 4), ..., a M _m Including. In addition, as illustrated in FIG. 8, the audio-video input / output device of the present embodiment further includes a target sound emphasizing unit 7 in addition to the components of the audio-video input / output device of the first embodiment.

The front sound pickup units M _{n + 1} ,..., M _m are microphones that pick up sound coming from a direction corresponding to the user's field of view when worn by the user. FIG. 7 shows an example in which the front sound pickup units M _{n + 1} ,..., M _m are arranged horizontally near the imaging unit V in the forehead, but the sound arriving from the direction corresponding to the user's visual field is shown. Any arrangement may be used as long as sound can be collected.

Target sound enhancement unit 7, the front sound pickup unit M _{n + 1,} ..., front audio signal x _{n + 1} M _m is picked up, respectively, ..., performs target sound enhancement process on x _m, specific sound Output the target sound emphasizing signal _{ao2 in} which. The target sound emphasis signal _ao2 is sent to the sound generation unit 4 and the sound output unit C1. The sound to be emphasized is specified by the user using voice command input, dial operation, and button operation. Specific specific operations will be described later. For the target sound emphasis processing, any known method may be applied. For example, the sound signal emphasis technique described in the above-mentioned Reference 1 can be used.

FIG. 9 is an image used when performing audio zoom in the audio / video input / output device of the present embodiment. The audio zoom is a function for collecting sound coming from a specific sound source in a concentrated manner. First, the user inputs "voice zoom" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the voice zoom function to the video generation unit 6 based on the voice recognition result op “voice zoom”. The video generation unit 6 displays the current audio zoom setting (center position and magnification) on the transmission screen of the video display unit G. In the example of FIG. 9, the center of the thick circle indicates the zoom position, and the radius of the circle indicates the magnitude of the zoom magnification. The user changes the audio zoom setting by dialing and button operation. First, when the dial operation unit D is rotated, the abscissa of the zoom position moves on the transmission screen. When the button operation unit B is pressed at a desired position, the abscissa is determined. Next, when the dial operation unit D is rotated, the ordinate of the zoom position moves on the transmission screen. When the button operation unit B is pressed at a desired position, the ordinate is determined. Thereafter, when the dial operation unit D is rotated, the zoom magnification changes. Finally pressing the button operation unit B, the control signal control signal c ₁ for determining the change of the audio zoom setting is sent to the target sound is emphasized section 7. The target sound emphasis unit 7 outputs a target sound emphasis signal _{ao2 in} which a sound coming from the specified direction is emphasized at a specified magnification. Audio zoom can be used to record video and audio from the wearer's point of view at construction sites or disaster sites. In particular, when the sound of a person or an object at a long distance is selectively distinguished under noise, the audio zoom setting can be performed in conjunction with the transmission screen, so that the operation can be performed intuitively and efficiently. Those instructions as described above, the wearer's eye position and a forward sound pickup unit M _{n + 1,} ..., can be realized because it is a helmet integrated device that the position relation of M _m is close and constant It is.

FIG. 10 is an image used when performing audio tracking in the audio-video input / output device of the present embodiment. First, the user inputs "voice tracking" by voice command input. The function control unit 3 sends a control signal c ₄ for calling the voice tracking function to the video generation unit 6 based on the voice recognition result op “voice tracking”. The image generation unit 6 displays on the transmission screen of the image display unit G while highlighting the tracking target candidate in the image in the image currently captured by the imaging unit V. In the example of FIG. 10, three objects are displayed as tracking target candidates, and an object located at the upper left of the transmissive screen is highlighted as a tracking target candidate. When the user rotates the dial operation unit D, a dial operation signal d is input to the function control unit 3, and a control signal c ₄ for switching the tracking target candidate is sent to the video generation unit 6. When the user selects a desired tracking target candidate by a dial operation and presses the button operation unit B in that state, a button operation signal b is input to the function control unit 3 and a control signal c ₁ for determining the tracking target. Is sent to the target sound emphasizing unit 7. Target sound enhancement unit 7 starts the tracked audio track determined in accordance with the control signal c _1. Thereafter, as long as the tracking target exists at the angle of view of the imaging unit V, the target sound emphasis signal _ao2 that emphasizes the sound coming from the direction of the tracking target is output. By combining the voice tracking and the voice zoom, it is also possible to specify the magnification of voice emphasis. Assuming a situation where it is difficult for the user to perform a dial operation, a tracking target can be selected by voice command input. A method is conceivable in which the user switches the tracking target candidate by inputting a voice command such as “right” or “left”, or selects the tracking target candidate by inputting a voice command directly on the screen. By selecting a tracking target by inputting a voice command, a completely hands-free operation becomes possible.

[Third embodiment]
In the first embodiment and the second embodiment, the sound generation unit 4 may be configured to have an external noise suppression function. In this case, the sound generation unit 4 obtains environmental noise around the sound emitting unit S using an existing sound collection unit, a front sound collection unit, or a dedicated sound collection unit, and generates a signal for suppressing the environmental noise. suppressing environmental noise by adding the output sound signal a _s to. With such a configuration, when utilized in noisy, there is an effect that the output acoustic signal a _s outputted from the sound emitting portion S is easily heard more for the user. For the external noise suppression processing, any known method may be applied, and for example, a noise suppression technique described in Reference Document 2 below can be used.
[Reference Document 2] JP-A-7-303135

  The points of the audio / video input / output device of the present invention are as follows. The first point is that a plurality of microphones for emphasizing the target sound are installed on the integrated helmet so that the wearer's utterance and surrounding sounds can be easily collected. The second feature is a transparent screen as an intuitive interface for controlling the mounted camera and multiple microphones, as well as voice command input, dial operation, and button operation functions, making video and audio control intuitive and efficient. This is the point that can be done.

  Since the audio-video input / output device of the present invention has the target sound emphasizing function, the voice of the wearer can be collected clearly even in a high-noise environment. As a result, the voice command input can be used stably as a means for communicating the intention of the wearer to the device. In addition, by using a helmet-integrated device equipped with a dial and a transmissive screen, parameters such as direction, emphasis rate, and magnification can be intuitively operated for photographing and sound pickup by a microphone and a camera. Furthermore, by using the integrated type, the wearer can perform work such as receiving instructions from a remote place while sharing moving images of the site with a communication partner while keeping both hands free. In particular, in a case where it is necessary to work while communicating with a remote instructor in a dangerous place such as a high noise place or a construction site, or while confirming electronic data, the audio / video input / output device of the present invention is used. It is possible to work safely, accurately, and efficiently by using.

  The present invention is not limited to the above embodiment, and it goes without saying that changes can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be performed not only in chronological order according to the order described, but also in parallel or individually according to the processing capability of the device that executes the process or as necessary.

[Program, recording medium]
When the various processing functions in each device described in the above embodiment are implemented by a computer, the processing content of the function that each device should have is described by a program. By executing this program on a computer, various processing functions of the above-described devices are realized on the computer.

  A program describing this processing content can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The distribution of the program is performed by, for example, selling, transferring, lending, or the like, a portable recording medium such as a DVD or a CD-ROM on which the program is recorded. Further, the program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, when executing the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of the program, the computer may directly read the program from the portable recording medium and execute processing according to the program, and further, the program may be transferred from the server computer to the computer. Each time, the processing according to the received program may be sequentially executed. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing instructions and obtaining results without transferring a program from the server computer to the computer. It may be. It should be noted that the program in the present embodiment includes information used for processing by the computer and which is similar to the program (data that is not a direct command to the computer but has characteristics that define the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of the processing may be realized by hardware.

M sound collection unit S sound emission unit V shooting unit G video display unit C communication recording unit D dial operation unit B button operation unit 1 input voice enhancement unit 2 voice recognition unit 3 function control unit 4 voice generation unit 5 video acquisition unit 6 video Generation unit 7 Target sound enhancement unit C1 Audio output unit C2 Audio input unit C3 Video output unit C4 Video input unit

Claims (2)

A plurality of sound collecting units for collecting surrounding sounds including a user's voice and environmental noise;
A photographing unit for photographing an area corresponding to the user's field of view,
An image display unit in which a screen is arranged at a position within the field of view of the user,
A dial operation unit that outputs a dial operation signal indicating a rotation direction and a rotation angle according to the rotation operation,
A button operation unit that is arranged at the position of the rotation axis on the surface of the dial operation unit and outputs a button operation signal indicating a pressed state,
A voice recognition unit that generates a voice recognition result by performing voice recognition on an acoustic signal obtained using the sound collection unit;
An image acquisition unit that acquires an image signal using the imaging unit,
A video generation unit that outputs an output video signal generated using the video signal to the video display unit,
Based on the dial operation signal, the button operation signal, and the voice recognition result, the video acquisition unit, the voice recognition unit, and a function control unit that generates a control signal that controls the function of the video generation unit, Including
The plurality of sound pickup units,
A first microphone placed at the mouth of the user to pick up the voice of the user and the environmental noise, and a position where the voice of the user is difficult to reach to pick up only the environmental noise And a second microphone arranged,
By alternately operating the dial operation unit and the button operation unit, it is possible to change the coordinates and range to be subjected to audio zoom among the images displayed on the image display unit,
The function control unit, when the voice recognition result is for calling a voice tracking function,
Highlighting a plurality of tracking target candidates included in the video signal on the display unit,
Switching the selection of the tracking target candidate based on the dial operation signal and the button operation signal based on the operation of the user or the voice command issued by the user, and the selected tracking target candidate as a tracking target,
An audio-video input / output device that continues to make sound arriving from a direction selected as the tracking target as an audio zoom target, as long as the tracking target exists at the angle of view of the imaging unit. The audio-video input / output device according to claim 1,
The second microphone is arranged in a direction opposite to a direction corresponding to the visual field of the user with the user as a center,
The plurality of sound collection units further include a third microphone arranged near the imaging unit to collect sound coming from a direction corresponding to the user's field of view,
Audio / video input / output device.

Images