JP2016206960A - Voice video input/output device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a smooth operation even in a high noise environment or in a use state in which a touch operation cannot be easily performed.SOLUTION: A voice recognition part 2 performs voice recognition to an acoustic signal obtained by acquiring sounds in the surrounding containing voice of a user using a sound acquisition part M, and creates a voice recognition result. A video acquisition part 5 acquires a video signal obtained by imaging an area corresponding ot a visual field of a user using an imaging part V. A voice creation part 4 outputs an output acoustic signal to a sound release part S. A video creation part 6 outputs an output video signal created by using the video signal to a video display part G. A function control part 3 creates a control signal for controlling functions of the video acquisition part 5, voice recognition part 2, voice creation part 4, and the video creation part 6 based on a dial operation signal from a dial operation part D, a button operation signal from a button operation part B, and the voice recognition result.SELECTED DRAWING: Figure 2

Description

  The present invention is an audio / video input / output device that can be operated by inputting an audio command.

  In recent years, there has been a growing need for factory workers who perform difficult or unskilled work to improve work efficiency by utilizing multimedia technology such as voice and video and information communication technology. However, carrying an information communication device such as a notebook computer or a tablet terminal separately from the tools necessary for work is a heavy burden on the worker. In such devices, keyboard operations and screen operations are mainly performed, but workers are not always in an environment where both hands can be freely used, and intuitive operability is often insufficient. For this reason, there is a problem that it is difficult to perform the work intended by the operator even if the work is performed in cooperation with the sound or the video. In addition, in such an environment, there is often a problem that noise is large and communication is often not performed smoothly.

  In response to the above-described needs, eyeglass-type wearable devices that display necessary information superimposed on an actual visual field have been developed. For example, there is Google Glass (registered trademark) described in Non-Patent Documents 1 and 2. Google Glass uses voice commands (for example, “OK glass”). For details, refer to Non-Patent Document 1.) and gesture commands (for example, finger touch and body tilt) For details, refer to Non-Patent Document 2).

Google, Inc., “Google Glass-Help-Voice actions”, [online], [Search April 3, 2015], Internet <URL: https://support.google.com/glass/answer/3079305? hl = en> Google, Inc., “Google Glass-Help-Glass gestures”, [online], [Search April 3, 2015], Internet <URL: https://support.google.com/glass/answer/3064184? hl = en>

  However, conventional glasses-type wearable devices are not intended for use in high-noise environments. For example, voices are buried in noise in environments where the surrounding noise is high, such as in factories, and command input by voice is likely to be erroneous. . In addition, workers in factories often work with thick gloves, and touchpads such as those equipped with conventional glasses-type wearable devices must be finely operated regardless of whether they are electrostatic or piezoelectric. Is difficult. In addition, there are many cases where an attachment to the head, such as a helmet, is required in the factory, and there is a case where a regular wearing method cannot be performed due to physical interference with these attachments. Furthermore, it is necessary to attach a fall prevention strap or the like in order to prevent a fall accident.

  An object of the present invention is to provide an audio / video input / output device capable of smooth operation even in a high noise environment or in a usage situation where touch operation is difficult.

  In order to solve the above-described problem, the audio / video input / output device according to the present invention captures a plurality of sound pickup units that pick up surrounding sounds including a user's voice and an area corresponding to the user's field of view. An imaging unit, a video display unit with a screen placed at a position that falls within the user's field of view, a dial operation unit that outputs a dial operation signal indicating a rotation direction and a rotation angle according to a rotation operation, and a surface of the dial operation unit A button operation unit that outputs a button operation signal that is placed at the position of the rotation axis and indicates a pressed state, a voice recognition unit that recognizes an acoustic signal acquired using the sound collection unit and generates a voice recognition result, and shooting Based on a video acquisition unit that acquires a video signal using a unit, a video generation unit that outputs an output video signal generated using the video signal to a video display unit, a dial operation signal, a button operation signal, and a voice recognition result And Image acquisition unit includes a voice recognition unit, and a function control unit for generating a control signal for controlling the function of the image generating unit.

  Since the audio / video input / output device of the present invention recognizes voice using acoustic signals collected from a plurality of sound collection units, voice command input operates stably even in a high noise environment. Further, since a physical operation with a dial and a button is possible, the operation is easy even in a usage situation where a touch operation is difficult. Therefore, smooth operation is possible even in high noise environments and usage situations where touch operations are difficult.

FIG. 1 is a diagram illustrating a functional configuration of the audio / video input / output device according to the first embodiment. FIG. 2 is a diagram illustrating a functional configuration of the audio / video input / output device according to the first embodiment. FIG. 3 is a diagram illustrating an example of use of the environment setting function. FIG. 4 is a diagram illustrating a usage image of the volume setting function. FIG. 5 is a diagram illustrating an example of a usage image of the video zoom function. FIG. 6 is a diagram illustrating an example of a usage image of the video tracking function. FIG. 7 is a diagram illustrating a functional configuration of the audio / video input / output device according to the second embodiment. FIG. 8 is a diagram illustrating a functional configuration of the audio / video input / output device according to the second embodiment. FIG. 9 is a diagram illustrating a usage image of the audio zoom function. FIG. 10 is a diagram illustrating a usage image of the voice tracking function.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

[First embodiment]
The first embodiment of the present invention is a helmet-integrated audio / video input / output device that is assumed to be used by a user wearing on the head at a high-noise work site such as in a factory. As shown in FIG. 1, the audio / video input / output device of the present embodiment includes n (≧ 2) sound collecting units M ₁ ,..., M _n , sound emitting unit S, photographing unit V, video display unit G, A dial operation unit D, a button operation unit B, and a communication recording unit C are included.

The sound pickup units M ₁ ,..., M _n are microphones that pick up sounds around the user when worn by the user. In the example of FIG. 1, a configuration is shown in which one sound collection unit M ₁ is arranged at the user's mouth, and n−1 sound collection units M ₂ ,..., M _n are arranged horizontally on the back of the head. However, any arrangement may be used as long as ambient sounds including the user's utterance can be collected.

  The sound emitting unit S is headphones arranged at a position corresponding to the user's ear when the user wears the sound emitting unit S. Headphones are generally prepared for both ears, but the sound emitting unit S of this embodiment is only required to be prepared for at least one ear, and is normally prepared for both ears. It may be.

  The imaging unit V is a video camera that captures an area corresponding to the user's field of view by setting an angle of view in the area corresponding to the user's field of view when the user wears it. In the example of FIG. 1, the configuration is shown in which the user's forehead is placed. However, the region corresponding to the user's field of view may be placed in any position as long as it can be photographed. However, it is desirable that the photographing unit V be arranged at a position that does not block the user's field of view.

  The video display unit G is a small display using a transmissive screen in which a screen is arranged at a position that enters the user's field of view when the user wears it. Since the video display unit G is a transmission type, the visual field of the user is not obstructed when there is no video output, and the video is superimposed on the front scene viewed from the user when the video output is present. Is done. Here, since it is assumed to be used in a work site such as a factory, there is an effect of protecting the eyes of the user from flying objects such as dust. The video display unit G is movable so that it can be retracted outside the range of the visual field of the user. For example, convenience is high when confirmation with the naked eye is necessary, for example, by confirming an accurate color. When retracted, it is desirable to store in a position that does not fall within the angle of view of the photographing unit V.

  The dial operation unit D is disposed on the surface of the sound emitting unit S that is opposite to the user's ear when the user wears the dial operation unit D, and is a large dial that is rotatable about the position of the user's ear It is. The dial operation part D has a plurality of irregularities formed on the side surface of the circular dial, and is formed so that it can be easily operated with a fingertip even when the user is wearing thick gloves, for example. Is done. The dial operation unit D outputs a dial operation signal d indicating the rotation direction and the rotation angle operated when the rotation operation is performed. The position of the dial operation part D does not necessarily have to be on the surface of the sound emitting part S, and may be arranged in any way as long as the user can operate it.

  The button operation unit B is a button that is disposed at the position of the rotation axis on the surface of the dial operation unit D and can be pressed in the direction of the center axis of the rotation dial. By setting the button operation unit B so that the cap portion is slightly depressed from the surface of the dial operation unit D even when the cap portion is not pressed, it is possible to prevent malfunctions due to unintended button presses. The button operation unit B outputs a button operation signal b indicating a pressed state when a press operation is performed.

The communication recording unit C establishes a wireless or wired communication path with a remote communication partner, and transmits and receives audio signals and video signals. In addition, the audio and video signals transmitted and received, the audio signals acquired using the sound pickup units M ₁ ,..., M _n and the video signals acquired using the photographing unit V are recorded on a recording medium or the like.

The operation of the audio / video input / output device of the first embodiment will be described with reference to FIG. The audio / video input / output device of this embodiment includes a sound collection unit M ₁ ,..., M _n , a sound emission unit S, a photographing unit V, a video display unit G, a dial operation unit D, a button operation unit B, and a communication recording unit C. In addition, an input voice emphasis unit 1, a voice recognition unit 2, a function control unit 3, a voice generation unit 4, a video acquisition unit 5, and a video generation unit 6 are included. As shown in FIG. 2, the communication recording unit C includes an audio output unit C1, an audio input unit C2, a video output unit C3, and a video input unit C4.

Input speech enhancement unit 1, the sound collection unit M _1, ..., acoustic signals x ₁ M _n is picked up, respectively, ..., performs target sound enhancement process on x _n, the acoustic signals x _1, ..., x _n The speech enhancement signal a _o in which the speech included in is enhanced is output. The voice enhancement signal a _o is sent to the voice recognition unit 2, the voice generation unit 4, and the voice output unit C1. The sound signal x ₁ picked up by the sound pickup unit M _{1 at the} user's mouth contains the user's voice and environmental noise, and the sound pickup unit M arranged at a position where the user's voice is difficult to reach. _The acoustic signals x ₂ ,..., X _n picked up by ₂ ,..., M _n can be expected to contain only environmental noise. Therefore, by suppressing the environmental noise contained in the acoustic signals x ₂ ,..., X _n in the acoustic signal x ₁ , it is possible to obtain an acoustic signal that emphasizes the user's voice. The target sound enhancement process is not limited to the above method, and any known method may be applied. For example, an acoustic signal enhancement technique described in Reference Document 1 below can be used.
[Reference Document 1] JP 2013-179388 A

The voice recognition unit 2 performs voice recognition on the voice enhancement signal a _o and outputs a voice recognition result op. Any known method may be applied to the voice recognition process. The voice recognition result op is sent to the function control unit 3. The voice recognition unit 2 does not have to execute the voice recognition process by itself, but transmits a voice enhancement signal a _o to a voice recognition device installed remotely, and a voice recognition result returned from the voice recognition device May be output as the speech recognition result op. At this time, communication with the speech recognition apparatus may be performed using the communication recording unit C.

Based on the dial operation signal d from the dial operation unit D, the button operation signal b from the button operation unit B, and the voice recognition result op from the voice recognition unit 2, the function control unit 3 Control signals c ₁ ,..., C ₅ for controlling. The content of the control signal is determined by the content selected by the user by dial operation, button operation, and voice command input according to the display on the video display unit G. Examples of operations for individual functions will be described later, but any combination of dial operation, button operation, and voice command input to configure the operation interface is arbitrary. For example, as a basic operation, a function may be called by calling a function by inputting a voice command, selecting an option by dialing, and confirming a selection by operating a button. It is also possible to collectively perform the process from designation of options to determination by inputting voice commands. The control signal is individually generated according to the component to be controlled. For example, the control signal c ₁ is a signal for controlling the input speech enhancement unit 1. The control signal c ₂ is a signal for controlling the sound generator 4. The control signal c ₃ is a signal for controlling the video acquisition unit 5. The control signal c ₄ is a signal that controls the video generation unit 6. The control signal c ₅ is a signal for controlling the voice recognition unit 2.

The voice output unit C1 transmits the voice enhancement signal _ao output from the input voice enhancement unit 1 to a remote communication partner. Alternatively, the voice enhancement signal a _o is stored in a recording medium not shown.

The voice input unit C2 receives a remote voice signal a _i from a remote communication partner. The received remote audio signal a _i is sent to the audio generator 4. The remote voice signal a _i is, for example, a voice for instructing the content of work to be performed by the remote communication partner to the user.

Sound generation unit 4, speech enhancement signal outputs of the input speech enhancement unit 1 a _o and the output to the remote audio signal a _i if there is a remote audio signal a _i and the output sound signal a _s using a speech input unit C2 , and outputs the output sound signal a _s to the sound emitting section S. The voice enhancement signal a _o is used, for example, to feed back a user's voice when the voice is used in a high noise environment and it is difficult to hear his / her voice. In addition, voices recorded in advance may be stored in the communication recording unit C and the like, and the voices may be reproduced and used as the remote voice signal a _i .

The video acquisition unit 5 acquires a video signal v _o captured using the imaging unit V. The acquired video signal v _o is sent to the video output unit C3.

The video output unit C3 transmits the video signal v _o output from the video acquisition unit 5 to a remote communication partner. Alternatively, the video signal v _o is stored in a recording medium (not shown).

The video input unit C4 receives a remote video signal v _i from a remote communication partner. The received remote video signal v _i is sent to the video generator 6. The remote video signal v _i is, for example, reference information required for work to be performed by the user.

Image generating unit 6, generates a video signal v _o output from the image capturing section 5, an output video signal v _s using the remote video signal v _i If the remote video signal v _i output from the image input unit C4 and outputs the output video signal v _s to the video display unit G.

FIG. 3 is a usage image when environment setting is performed in the audio / video input / output device of this embodiment. First, the user inputs “environment” by voice command input. The function control unit 3 sends a control signal c ₄ for calling the environment setting function to the video generation unit 6 based on the voice recognition result op “environment”. The video generation unit 6 displays the current environment setting on the transparent screen of the video display unit G. In the example of FIG. 3, the current environment setting is “construction site”, and “tunnel”, “server room”, and the like are displayed as other setting candidates. These options and the operation parameters associated therewith are set in advance. When the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3 and a control signal c ₄ for changing the environment setting candidate is sent to the video generation unit 6. And presses the button operation unit B after selecting the desired configuration candidate by dial operation, button operation signal b is input to the function control unit 3, the control signals c ₁ and c ₅ for confirming the change of configuration It is sent to the input speech enhancement unit 1 and the speech recognition unit 2. The input speech enhancement unit 1 and the speech recognition unit 2 change the operation parameters according to the control signals c ₁ and c ₅ . Since target sound enhancement processing and speech recognition processing generally have different optimum operating parameters depending on the environment, it can be expected that more accurate processing results can be obtained by correctly setting the usage environment.

FIG. 4 is a usage image when performing volume setting in the audio / video input / output device of this embodiment. First, the user inputs “volume” by voice command input. The function control unit 3 sends a control signal c ₄ for calling the volume setting function to the video generation unit 6 based on the voice recognition result op “volume”. The video generation unit 6 displays the current volume setting on the transparent screen of the video display unit G. In the example of FIG. 4, the current volume setting is “50”. When the user rotates the dial operation unit D, dialing signal d is input to the function control unit 3, the control signal c ₂ for changing the volume setting to the sound generating unit 4, for changing the volume display A control signal c ₄ is sent to the video generator 6. Sound generation unit 4 to lower the volume of the output sound signal a _s to be output to the sound emitting unit S according to the control signal c _2. Image generation unit 6 to lower the volume display in accordance with the control signal c _4.

FIG. 5 is a usage image when video zooming is performed in the audio / video input / output device of this embodiment. First, the user inputs “video zoom” by voice command input. The function control unit 3 sends a control signal c ₄ for calling the video zoom function to the video generation unit 6 based on the voice recognition result op “video zoom”. The video generation unit 6 displays the video currently captured by the imaging unit V on the transmission screen of the video display unit G. When the user rotates the dial operation unit D, a dial operation signal d is input to the function control unit 3, and a control signal c ₃ for changing the magnification of the image zoom is sent to the image acquisition unit 5. The video acquisition unit 5 acquires the video signal v _o by changing the magnification of the photographing unit V. When shooting a video with hands-free, there is a problem that the photographer does not know the subject accurately with a single camera attached overhead or the like, and it is difficult to perform an operation of directly touching the photographing unit V. If the image zoom can be performed by the operation as described above, the image range can be confirmed and the image zoom operation can be performed smoothly.

FIG. 6 shows a usage image when video tracking is performed in the audio / video input / output device of this embodiment. First, the user inputs “video tracking” by voice command input. The function control unit 3 sends a control signal c ₄ for calling the video tracking function to the video generation unit 6 based on the voice recognition result op of “video tracking”. The video generation unit 6 displays on the transparent screen of the video display unit G while highlighting the tracking target candidate in the video in the video currently captured by the imaging unit V. In the example of FIG. 6, three objects are displayed as tracking target candidates, and an object located at the upper left of the transparent screen is highlighted as a tracking target candidate. Here, when the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and a control signal c ₄ for switching the tracking target candidate is transmitted to the video generation unit 6. When the user selects a desired tracking target candidate by dialing and presses the button operation unit B in that state, the button operation signal b is input to the function control unit 3 and a control signal c ₃ for determining the tracking target. Is sent to the video acquisition unit 5. Video acquisition unit 5 starts video tracking of the tracking target determined in accordance with the control signal c _3. Thereafter, as long as the tracking target exists at the angle of view of the imaging unit V, the video signal _vo centered on the tracking target is captured. By combining video tracking and video zoom, it is possible to track any object in the field of view while zooming. In this case, the photographing unit V needs to support a pan / tilt / zoom function. It is also possible to select a tracking target by inputting a voice command, assuming that it is difficult for the user to perform a dial operation. A method is conceivable in which the user switches the tracking target candidate by inputting a voice command such as “right” or “left”, or selects the tracking target candidate by directly inputting a voice command of coordinates on the screen. By selecting a tracking target by inputting a voice command, a completely hands-free operation becomes possible.

[Second Embodiment]
As illustrated in FIG. 7, the audio / video input / output device of the second embodiment includes n (≧ 2) sound collecting units M ₁ ,..., M _n , sound emitting unit S, photographing unit V, and video display unit. G, dial operation unit D, the button operations part B, and includes a communication recording portion C as in the first embodiment, the front sound pickup unit M _{n + 1} of the further mn number (m ≧ 4), ..., a M _m Including. In addition to the components of the audio / video input / output device of the first embodiment, the audio / video input / output device of this embodiment further includes a target sound enhancement unit 7 as illustrated in FIG.

The front sound pickup units M _{n + 1} ,..., M _m are microphones that pick up sounds coming from the direction corresponding to the user's visual field when the user wears them. FIG. 7 shows an example in which the front sound pickup units M _{n + 1} ,..., M _m are horizontally arranged in the vicinity of the imaging unit V in the forehead, but the sound coming from the direction corresponding to the user's field of view. As long as sound can be collected, any arrangement may be used.

Target sound enhancement unit 7, the front sound pickup unit M _{n + 1,} ..., front audio signal x _{n + 1} M _m is picked up, respectively, ..., it performs target sound enhancement process on x _m, specific sound The target sound emphasizing signal a _o2 in which is emphasized is output. The target sound enhancement signal a _o2 is sent to the voice generation unit 4 and the voice output unit C1. The sound to be emphasized is specified by the user using voice command input, dial operation, and button operation. A specific specific operation will be described later. For the target sound enhancement process, any known method may be applied. For example, the acoustic signal enhancement technique described in the above-mentioned Reference 1 can be used.

FIG. 9 is a usage image when performing audio zoom in the audio / video input / output device of the present embodiment. The audio zoom is a function that collects sounds arriving from a specific sound source in a concentrated manner. First, the user inputs “voice zoom” by voice command input. The function control unit 3 sends a control signal c ₄ for calling the audio zoom function to the video generation unit 6 based on the audio recognition result op “audio zoom”. The video generation unit 6 displays the current audio zoom setting (center position and magnification) on the transmission screen of the video display unit G. In the example of FIG. 9, the center of the thick circle indicates the zoom position, and the radius of the circle indicates the zoom magnification. The user changes the audio zoom setting by dial operation and button operation. First, when the dial operation unit D is rotated, the abscissa of the zoom position moves on the transmission screen. When the button operation unit B is pressed at a desired position, the abscissa is determined. Next, when the dial operation unit D is rotated, the ordinate of the zoom position moves on the transmission screen. When the button operation unit B is pressed at a desired position, the ordinate is determined. Thereafter, when the dial operation unit D is rotated, the zoom magnification changes. Finally, when the button operation unit B is pressed, a control signal control signal c ₁ for confirming the change of the audio zoom setting is sent to the target sound emphasizing unit 7. The target sound emphasizing unit 7 outputs a target sound emphasizing signal a _{o2 in} which the sound coming from the specified direction is emphasized at a specified magnification. Audio zoom can be used to record video and audio from the viewpoint of the wearer at construction sites or disaster sites. In particular, when the sound of a long-distance person or object is selectively recognized under noise, the sound zoom setting can be performed in conjunction with the transmissive screen, thereby enabling intuitive and efficient operation. The above-described operation method can be realized because it is a helmet-integrated device in which the positional relationship between the wearer's eye position and the front sound pickup units M _{n + 1} ,..., M _m is close and constant. It is.

FIG. 10 is a usage image when performing audio tracking in the audio / video input / output device of this embodiment. First, the user inputs “voice tracking” by voice command input. The function control unit 3 sends a control signal c ₄ for calling the voice tracking function to the video generation unit 6 based on the voice recognition result op “voice tracking”. The video generation unit 6 displays on the transparent screen of the video display unit G while highlighting the tracking target candidate in the video in the video currently captured by the imaging unit V. In the example of FIG. 10, three objects are displayed as tracking target candidates, and an object located at the upper left of the transparent screen is highlighted as a tracking target candidate. When the user rotates the dial operation unit D, the dial operation signal d is input to the function control unit 3, and a control signal c ₄ for switching the tracking target candidate is sent to the video generation unit 6. When the user selects a desired tracking target candidate by a dial operation and presses the button operation unit B in that state, the button operation signal b is input to the function control unit 3 and a control signal c ₁ for determining the tracking target. Is sent to the target sound enhancement unit 7. Target sound enhancement unit 7 starts the tracked audio track determined in accordance with the control signal c _1. Thereafter, as long as the tracking target exists at the angle of view of the imaging unit V, the target sound enhancement signal a _o2 in which the sound coming from the direction of the tracking target is emphasized is output. By combining voice tracking and voice zoom, it is also possible to specify the voice enhancement magnification. It is also possible to select a tracking target by inputting a voice command, assuming that it is difficult for the user to perform a dial operation. A method is conceivable in which the user switches the tracking target candidate by inputting a voice command such as “right” or “left”, or selects the tracking target candidate by directly inputting a voice command of coordinates on the screen. By selecting a tracking target by inputting a voice command, a completely hands-free operation becomes possible.

[Third embodiment]
In 1st embodiment and 2nd embodiment, you may comprise so that the audio | voice production | generation part 4 may be provided with an external noise suppression function. In this case, the voice generation unit 4 acquires the environmental noise around the sound emission unit S using the existing sound collection unit, the front sound collection unit, or the dedicated sound collection unit, and generates a signal for suppressing the environmental noise. suppressing environmental noise by adding the output sound signal a _s to. With such a configuration, when utilized in noisy, there is an effect that the output acoustic signal a _s outputted from the sound emitting portion S is easily heard more for the user. Any known method may be applied to the external noise suppression processing. For example, a noise suppression technique described in Reference Document 2 below can be used.
[Reference Document 2] Japanese Patent Laid-Open No. 7-303135

  The points of the audio / video input / output device of the present invention are as follows. The first point is that multiple microphones for emphasizing the target sound are installed in the integrated helmet so that the wearer's speech and surrounding sounds can be easily collected. The second is an intuitive interface for controlling the installed camera and multiple microphones, including a transparent screen, voice command input, dial operation, and button operation functions, making video and audio control intuitive and efficient. This is the point that can be done.

  Since the audio / video input / output device of the present invention has a target sound enhancement function, the voice of the wearer can be clearly picked up even in a high noise environment. As a result, the voice command input can be stably used as a means for transmitting the wearer's intention to the apparatus. Further, by adopting a helmet-integrated device equipped with a dial and a transmissive screen, parameters such as direction, enhancement rate, and magnification can be intuitively operated for shooting and sound collection by a microphone and a camera. Furthermore, by adopting the integrated type, the wearer can perform operations such as receiving instructions from a remote location while freeing both hands and sharing a video on the site with the communication partner. The audio / video input / output device according to the present invention, particularly in cases where it is necessary to perform work while interacting with a remote instructor or checking electronic data in a dangerous place such as a noisy place or a construction site. It is possible to work safely, accurately and efficiently.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

M sound collecting part S sound emitting part V photographing part G video display part C communication recording part D dial operation part B button operation part 1 input voice emphasis part 2 voice recognition part 3 function control part 4 voice generation part 5 video acquisition part 6 video Generation unit 7 Target sound enhancement unit C1 Audio output unit C2 Audio input unit C3 Video output unit C4 Video input unit

Claims (8)

A plurality of sound collection units for collecting surrounding sounds including user's voice;
An imaging unit for imaging an area corresponding to the user's field of view;
A video display unit in which a screen is arranged at a position within the user's field of view;
A dial operation unit that outputs a dial operation signal indicating a rotation direction and a rotation angle according to the rotation operation;
A button operation unit that outputs a button operation signal that is disposed at the position of the rotation axis on the surface of the dial operation unit and indicates a pressed state;
A speech recognition unit that recognizes an acoustic signal acquired using the sound collection unit and generates a speech recognition result; and
A video acquisition unit that acquires a video signal using the imaging unit;
A video generation unit that outputs an output video signal generated using the video signal to the video display unit;
A function control unit that generates control signals for controlling the functions of the video acquisition unit, the voice recognition unit, and the video generation unit, based on the dial operation signal, the button operation signal, and the voice recognition result;
Audio video input / output device. The audio / video input / output device according to claim 1,
An input speech enhancement unit that emphasizes speech included in the acoustic signal and outputs the speech to the speech recognition unit;
An audio / video input / output device further comprising: The audio / video input / output device according to claim 1 or 2,
The audio / video input / output device, wherein the video acquisition unit generates the video signal obtained by enlarging or / and tracking an area designated at an angle of view of the photographing unit based on the control signal. The audio / video input / output device according to any one of claims 1 to 3,
A sound emitting unit disposed at a position corresponding to the user's ear;
A plurality of front sound collection units that collect sound coming from a direction corresponding to the user's field of view;
A target sound emphasizing unit that emphasizes a specific sound included in the front acoustic signal acquired using the front sound collecting unit;
A sound generation unit that outputs an output sound signal generated using the sound signal and the front sound signal to the sound emitting unit;
An audio / video input / output device further comprising: The audio / video input / output device according to claim 4,
The function control unit generates a control signal for controlling the function of the target sound enhancement unit,
The audio / video input / output device according to claim 1, wherein the target sound emphasizing unit emphasizes sound coming from a sound source existing in a direction specified in the angle of view of the photographing unit based on the control signal. The audio / video input / output device according to claim 4 or 5,
The audio / video input / output apparatus, wherein the audio generation unit acquires environmental noise around the sound emitting unit using the sound collection unit, and adds a signal for suppressing the environmental noise to the output acoustic signal. The audio / video input / output device according to any one of claims 1 to 6,
The audio / video input / output device is capable of moving the video display unit to a position outside the range of the field of view of the user and outside the range of the angle of view of the photographing unit. The audio / video input / output device according to any one of claims 1 to 7,
The voice recognition unit transmits the acoustic signal to a voice recognition device using a communication unit, and receives a voice recognition result obtained by voice recognition of the acoustic signal by the voice recognition device using the communication unit. Video input / output device.

Images