JP2019102062A - Head-mounted display and method for controlling the same - Google Patents

Abstract

To provide a head-mounted display that allows a wearer of the head-mounted display and a third party to have a smooth communication to such an extent that both can understand each other's intention, and a method for controlling the same.SOLUTION: A head-mounted display 10 comprises: a sound pick-up unit 11 that picks up a sound from a specific direction; a voice detection unit 14 that detects a voice from the sound picked up by the sound pick-up unit 11; a head movement detection unit 15 that detects the movement of the head of a wearer of the head-mounted display 10; an external display 16 that displays information that can be visually recognized from the outside; and a response detection unit 17 that, when the voice is detected by the voice detection unit 14 and a movement of the head of the wearer being turned in a specific direction is detected by the head movement detection unit 15, displays information indicating that the wearer responds to the voice on the external display 16.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to a head mounted display and a control method thereof.

  A user wearing a head-mounted display (hereinafter, also referred to as a "wearer") can immerse in video and audio while making it difficult to communicate with an external person.

  Therefore, conventionally, a technology has been proposed for facilitating communication between the wearer of the head mounted display and an external person (see Patent Document 1).

International Publication No. 2014/156388

  However, in the technology of Patent Document 1, although communication between the wearer of the head mounted display and an external person is facilitated to some extent, there is a problem that smooth communication can not be performed to such an extent that both intentions can be confirmed.

  Thus, the present disclosure provides a head mounted display that enables smooth communication between the wearer of the head mounted display and an external person to confirm the intentions of both parties and a control method thereof.

  A head mounted display according to an embodiment of the present disclosure is a head mounted display that presents an image, and an audio from a sound pickup unit that collects sound from a specific direction and sound collected by the sound pickup unit. A voice detection unit for detecting the head movement, a head movement detection unit for detecting the movement of the head of the wearer of the head mounted display, an external display for displaying information visible from the outside, and the voice detection unit The information indicating that the wearer has responded to the voice when the head movement detection unit detects that the head movement of the wearer is directed to the specific direction is detected. And a response detection unit for displaying on an external display.

  A control method of a head mounted display according to an aspect of the present disclosure is a control method of a head mounted display that presents an image, and includes a sound collecting step of collecting sound from a specific direction, and collecting the sound in the sound collecting step. A voice detection step for detecting voice from the sound sound, a head movement detection step for sensing movement of a head of a wearer of the head mounted display, a voice detection step by the voice detection step, and the head When the movement detecting step detects that the wearer's head is directed to the specific direction, the information indicating that the wearer has responded to the voice can be viewed from outside. And a response detection step displayed on the external display to be displayed.

  A head mounted display according to an embodiment of the present disclosure is a head mounted display that presents an image, and an audio from a sound pickup unit that collects sound from a specific direction and sound collected by the sound pickup unit. Voice is detected by the voice detection unit, the body operation detection unit that detects the direction of the body of the wearer of the head mounted display, the external display that displays information visible from the outside, and the voice detection unit And displaying, on the external display, information indicating that the wearer has responded to the voice when it is detected by the body motion detection unit that the body of the wearer has turned in the specific direction. And a response detection unit.

  A control method of a head mounted display according to an aspect of the present disclosure is a control method of a head mounted display that presents an image, and includes a sound collecting step of collecting sound from a specific direction, and collecting the sound in the sound collecting step. A voice detection step for detecting voice from the sound sound, a body motion detection step for sensing the direction of the body of the wearer of the head mount display, a voice detection step by the voice detection step, and the body motion detection When the step detects that the body of the wearer has turned to the specific direction, information indicating that the wearer responds to the voice is displayed on an external display that displays information that can be viewed from the outside And displaying a response detection step.

  A head mounted display according to an embodiment of the present disclosure is a head mounted display that presents an image, and an audio from a sound pickup unit that collects sound from a specific direction and sound collected by the sound pickup unit. A voice detection unit that detects the movement of a hand or arm of the wearer of the head mounted display, an external display that displays information visible from outside, and the voice detection unit A response that displays information indicating that the wearer has responded to the voice on the external display, when detected and when the predetermined movement of the wearer's hand or arm is detected by the hand and arm motion detection unit And a detection unit.

  A control method of a head mounted display according to an aspect of the present disclosure is a control method of a head mounted display that presents an image, and includes a sound collecting step of collecting sound from a specific direction, and collecting the sound in the sound collecting step. A voice detection step for detecting voice from the sound sound, a hand and arm movement detection step for sensing movement of a hand or arm of the wearer of the head mounted display, voice is detected by the voice detection step, and the hand and arm When a predetermined movement of the wearer's hand or arm is detected by the motion detection unit, information indicating that the wearer has responded to the voice is displayed on an external display that displays externally visible information. And detecting the response.

  Note that these general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer readable CD-ROM, a system, a method, an integrated circuit, a computer program And any combination of recording media.

  According to the present disclosure, there is provided a head mounted display and a control method thereof that enable a wearer of the head mounted display and an external person to smoothly communicate with each other to confirm the intentions of both.

FIG. 1 is an external view of a head mounted display according to the embodiment. FIG. 2 is a block diagram showing the configuration of the head mounted display shown in FIG. FIG. 3 is a flowchart showing the operation of the head mounted display according to the embodiment. FIG. 4 is a block diagram showing the configuration of a head mounted display according to a first modification of the embodiment. FIG. 5 is a flowchart showing the operation of the head mounted display according to the first modification of the embodiment. FIG. 6 is a block diagram showing the configuration of a head mounted display according to a second modification of the embodiment. FIG. 7 is a flowchart showing the operation of the head mounted display according to the second modification of the embodiment. FIG. 8 is a block diagram showing a configuration of a head mounted display according to Modification 3 of the embodiment. FIG. 9 is a flowchart showing the operation of the head mounted display according to the third modification of the embodiment. FIG. 10A is an external view of a driving device configuring a remote control system according to Modification 4 of the embodiment. FIG. 10B is an external view of a car constituting the remote control system according to the fourth modification of the embodiment. FIG. 11 is a block diagram showing the main configuration of a remote control system according to Modification 4 of the embodiment. FIG. 12 is a block diagram showing a configuration of a head mounted display according to Modification 5 of the embodiment. FIG. 13 is a flowchart showing the operation of the head mounted display according to the fifth modification of the embodiment. FIG. 14 is a block diagram showing the configuration of a head mounted display according to Variation 6 of the embodiment. FIG. 15 is a flowchart showing the operation of the head mounted display according to the sixth modification of the embodiment. FIG. 16 is a diagram showing a display example on the external display of the head mounted display according to the sixth modification of the embodiment. FIG. 17 is a block diagram showing a configuration of a head mounted display according to Modification 7 of the embodiment. FIG. 18 is a flowchart showing the operation of the head mounted display according to the seventh modification of the embodiment. FIG. 19 is a block diagram showing the configuration of a head mounted display according to Variation 8 of the embodiment. FIG. 20 is a flowchart showing the operation of the head mounted display according to the eighth modification of the embodiment.

(Findings that formed the basis of this disclosure)
BACKGROUND In recent years, head-mounted displays, which are display devices worn on the head of a user and causing the user to view video and audio, have become widespread. As an application of the head mounted display, for example, as an in-flight service of an aircraft, an airline that provides in-flight entertainment to the passengers using the head mounted display has also appeared.

  A passenger wearing the head mounted display can immerse in video and audio, but has a problem that it is difficult to communicate with an external person. For example, a flight attendant who speaks to a passenger can not recognize whether the passenger is aware of himself or herself.

  Therefore, it is said that the passenger, while wearing the head mounted display, recognizes the cabin attendant approaching him and wants to make a simple communication with the cabin attendant while viewing the content without removing the head mounted display as much as possible. There is a demand. In addition, there is a demand that a cabin attendant wants to recognize that a passenger wearing a head mounted display is aware of himself.

  Here, as a technique for facilitating communication between a wearer of the head mounted display and an external person, a technique disclosed in Patent Document 1 has been proposed. In this technique, after notifying the wearer of the head mounted display (notification by video and audio), the effect is displayed on the outside of the head mounted display. This display allows the outsider to know that the wearer has been notified.

  However, in the technology of Patent Document 1, the fact that the wearer of the head mounted display has been notified is displayed on the outside of the head mounted display, and the wearer of the head mounted display necessarily notices the notification. The effect is not displayed on the outside of the head mounted display. For example, when the wearer is notified by voice and the volume of the content reproduced on the head mounted display is high, the wearer does not notice the notification. Even in such a case, in the technique of Patent Document 1, the effect of notifying the wearer is displayed on the outside of the head mounted display. That is, according to the technology of Patent Document 1, the outsider does not know whether the wearer has noticed the notification.

  Moreover, the technique of patent document 1 presupposes that the wearer of a head mounted display notices notification. In order to ensure that, when the notification to the wearer is a notification by video, a video that is easy for the wearer to notice is displayed. Therefore, the content viewing of the head mounted display is disturbed. On the other hand, when the notification to the wearer is a voice notification, the voice of the content being reproduced once on the head mounted display may be stopped or reduced to provide a voice that the wearer can easily notice. Processing is required. For that reason, too, the content viewing of the head mounted display is disturbed.

  Furthermore, in the technology of Patent Document 1, the intention of the wearer of the head mounted display, that is, whether or not to communicate with an external person is not reflected in the display on the outside of the head mounted display.

  As described above, according to the technology of Patent Document 1, although communication between the wearer of the head mounted display and an external person becomes possible to some extent, it is impossible to perform smooth communication to such an extent that both intentions can be confirmed. There's a problem.

  Thus, the present disclosure provides a head mounted display that enables smooth communication between the wearer of the head mounted display and an external person to confirm the intentions of both parties and a control method thereof.

  For that purpose, in the head mounted display according to an embodiment of the present disclosure, the position (sound coordinates) of an external person speaking to the wearer of the head mounted display is specified. Then, when it is detected that the wearer of the head mounted display has moved the head, body, hand or arm in the direction of the sound coordinates, it is judged that the wearer of the head mounted display has responded to the external voice, and the head This is indicated on the outside of the mount display.

  Thereby, it can be recognized from the outside that the wearer of the head mounted display has noticed the notification. Therefore, notification of not disturbing the content, that is, natural notification while watching the content can be performed to the wearer of the head mounted display. In addition, the intention of the wearer of the head mounted display, that is, whether or not to communicate with the outside is reflected in the display on the outside of the head mounted display.

  That is, according to the present disclosure, a head mounted display is realized that enables smooth communication that enables the wearer of the head mounted display and an external person to confirm and agree with each other.

  More specifically, a head mounted display according to an embodiment of the present disclosure is a head mounted display that presents an image, and a sound collecting unit that collects sound from a specific direction, and the sound collecting unit collect sound Voice detection unit for detecting voice from noise, head movement detection unit for sensing movement of the head of the wearer of the head mounted display, an external display for displaying information visible from outside, the voice detection A voice is detected by the unit, and the head movement detecting unit detects that the wearer responds to the voice when a motion is detected such that the head of the wearer is directed in the specific direction. And a response detection unit that displays the information to be displayed on the external display.

  This indicates that the wearer responds to the voice when it is detected that the head of the wearer of the head mounted display faces a specific direction set to the sound source direction of the voice emitted by an external person Information is displayed on the external display. Therefore, when the wearer shows the intention to respond to the voice of the external person, that is displayed on the external display, so that the smoothness of the degree that the wearer and the external person can confirm both intentions with each other Communication is possible.

  Here, the head mounted display further includes a speaker identification unit that determines whether the sound detected by the sound detection unit is a predetermined person's sound, and the response detection unit The speaker identification unit determines that the voice is a predetermined voice of a person, and the head movement detection unit detects a motion such that the head of the wearer points in the specific direction. In some cases, the information may be displayed on the external display.

  Thereby, when the wearer of the head mounted display indicates the intention to respond to the voice from a predetermined person, that is displayed on the external display. Therefore, it is possible to suppress false recognition that the wearer responds to the voice from an unrelated person as noise jumping into the wearer, and the response accuracy of the head mounted display is improved.

  The head mounted display may further include a sound source searching unit for searching a sound source, and the sound collecting unit may collect the direction of the sound source searched by the sound source searching unit as the sound from the specific direction. It is also good.

  As a result, since the direction of the sound source is searched by the sound source search unit, it is not necessary for the voice of an external person to be fixed to the sound from the specific direction. That is, when the wearer responds to the sound from all directions, that is displayed on the external display, and the convenience of the head mounted display is improved.

  Further, the head mount display further includes a headphone that provides the wearer with the sound accompanying the video, and an audio that superimposes the audio detected by the audio detection unit on the audio according to the video and outputs the audio to the headphone And a superimposing unit.

  As a result, the voice of the external person is superimposed on the sound accompanying the video and output from the headphone, so the wearer of the head mount display can listen to the voice of the external person while watching the audio of the content being reproduced. You can also listen. Therefore, more smooth communication is possible between the wearer and the outsider.

  Further, the sound superimposing unit may perform sound image localization so that the sound detected by the sound detecting unit can be heard from a direction corresponding to the specific direction, and then may be superimposed on the sound accompanying the video.

  As a result, the wearer of the head mounted display hears the voice of an external person from the actual sound source direction or the corresponding sound source direction according to sound image localization, and thus the head mount display is mounted similarly to the case where the head mounted display is not mounted. , You can easily detect the direction of the outside person who emitted the voice.

  In addition, the head mounted display further includes a headphone that provides the wearer with a sound accompanying the video, and the response detection unit is provided to the headphone when the information is displayed on the external display. You may turn down the volume of the sound.

  This reduces the volume of the sound provided to the headphones when an external person emits sound toward the wearer. Therefore, the wearer of the head mounted display can hear the voice of an external person while wearing the head mounted display.

  Further, in the response detecting unit, when the voice detecting unit detects voice and the head movement detecting unit detects a motion in which the head of the wearer faces the specific direction, Control may be performed to change the direction in which the sound collection unit collects sound so as to collect the sound emitted by the wearer.

  As a result, after the wearer responds to the voice from the external person, the sound collection direction is changed so as to pick up the voice from the wearer, so that the voice emitted by the wearer is It is picked up. Therefore, more smooth communication is possible between the wearer and the outsider.

  In addition, the other form of this indication is the control method of a head mounted display. The control method is a control method of a head mounted display that presents an image, which is a sound collecting step of collecting a sound from a specific direction, and an audio of detecting sound from the sound collected in the sound collecting step. Voice is detected by the detecting step, the head movement detecting step of detecting the movement of the head of the wearer of the head mounted display, and the voice detecting step, and the head movement detecting step detects the head of the wearer. A response detection step of displaying information indicating that the wearer has responded to the voice on an external display that displays information that can be visually recognized from the outside, when movement is detected such that the unit faces the specific direction And.

  In addition, a head mounted display according to an embodiment of the present disclosure is a head mounted display that presents an image, and a sound collecting unit collecting sound from a specific direction, and the sound collected by the sound collecting unit. The voice detection unit detects voice from the body, the body motion detection unit detects the direction of the body of the wearer of the head mounted display, the external display displays information visible from outside, and the voice detection unit When the body motion detection unit detects that the body of the wearer is facing the particular direction, information indicating that the wearer responds to the voice is displayed on the external display. And a response detection unit for displaying.

  Thereby, when it is detected that the body of the wearer of the head mounted display faces a specific direction set to the sound source direction of the voice emitted by the external person, information indicating that the wearer responds to the voice Is displayed on the external display. Therefore, when the wearer shows the intention to respond to the voice of the external person, that is displayed on the external display, so that the smoothness of the degree that the wearer and the external person can confirm both intentions with each other Communication is possible.

  In addition, the other form of this indication is the control method of a head mounted display. The control method is a control method of a head mounted display that presents an image, which is a sound collecting step of collecting a sound from a specific direction, and an audio of detecting sound from the sound collected in the sound collecting step. A voice is detected by the detection step, the body movement detection step of detecting the direction of the wearer's body of the head mount display, the voice detection step, and the body of the wearer is detected by the body movement detection step. And a response detecting step of displaying information indicating that the wearer has responded to the voice when it is detected that the user has turned in a specific direction, on an external display that displays information that can be visually recognized from the outside.

  In addition, a head mounted display according to an embodiment of the present disclosure is a head mounted display that presents an image, and a sound collecting unit collecting sound from a specific direction, and the sound collected by the sound collecting unit. A voice detection unit that detects voice from the body, a hand and arm operation detection unit that detects the movement of the hand or arm of the wearer of the head mounted display, an external display that displays information visible from outside, and the voice detection unit When the voice is detected, and the predetermined movement of the wearer's hand or arm is detected by the hand and arm motion detection unit, information indicating that the wearer responds to the voice is displayed on the external display And a response detection unit.

  Thereby, when a predetermined movement of the hand or arm of the wearer of the head mounted display is detected, information indicating that the wearer responds to the voice is displayed on the external display. Therefore, when the wearer shows the intention to respond to the voice of the external person, that is displayed on the external display, so that the smoothness of the degree that the wearer and the external person can confirm both intentions with each other Communication is possible.

  In addition, the other form of this indication is the control method of a head mounted display. The control method is a control method of a head mounted display that presents an image, which is a sound collecting step of collecting a sound from a specific direction, and an audio of detecting sound from the sound collected in the sound collecting step. A voice is detected by the detection step, a hand and arm motion detection step of sensing the movement of the wearer's hand or arm of the head mount display, and the voice detection step, and the hand or arm of the wearer is detected by the hand and arm motion detection step. And a response detection step of displaying information indicating that the wearer has responded to the voice when a predetermined movement of the arm is detected, on an external display that displays information that can be visually recognized from the outside.

  In addition, another aspect of the present disclosure is a recording medium such as a computer readable disk or a semiconductor memory in which a program or a program executed by a CPU, a processor or a computer is recorded. The program includes the step of configuring the control method of the head mounted display.

  Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that all the embodiments described below indicate general or specific examples of the present disclosure. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like described in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components not described in the independent claim indicating the highest concept of the present disclosure are described as optional components. Moreover, each figure is not necessarily illustrated exactly. In the drawings, substantially the same components are denoted by the same reference numerals, and redundant description will be omitted or simplified.

Embodiment
FIG. 1 is an external view of a head mounted display 10 according to the embodiment. The head mounted display 10 is a head mounted display that enables a wearer of the head mounted display 10 to communicate with an external person while confirming the intentions of both parties while wearing the head mounted display 10. is there. The head mounted display 10 is provided with microphones 12 a to 12 d, an external display 16 and headphones 35 as components seen from the outside.

  The microphones 12 a to 12 d are microphones that detect the sound source direction of the sound coming from the front of the head mounted display 10, and are provided at four places in the front, rear, top, and bottom of the head mounted display 10. The head mounted display 10 is not necessarily required to have four microphones. For example, two microphones may be provided apart only in the vertical direction or only in the horizontal direction.

  The external display 16 is a display that displays information that can be viewed from the outside, and is, for example, a lamp such as an LED, or a two-dimensional display such as an LCD. The “visible information from outside” may be simply lighting of a lamp, characters, symbols, icons, images, and a mixture of at least two of them.

  The headphone 35 is a headphone that provides the wearer with a sound accompanying the image provided by the head mounted display 10, and is an open air type or closed type headphone. In the present embodiment, the headphones 35 output sound associated with the image reproduced by the head mounted display 10, and an open air type headphone is also provided so that the sound of an external person can also be heard by the wearer of the head mounted display 10. Is preferred.

  FIG. 2 is a block diagram showing the configuration of the head mounted display 10 shown in FIG. The head mounted display 10 includes a sound collection unit 11, an audio detection unit 14, a head movement detection unit 15, an external display 16, a response detection unit 17, a video reproduction unit 30, an internal display 34, and headphones 35.

  The sound collection unit 11 is a processing unit that collects sound from a specific direction, and includes four microphones 12 a to 12 d and a directivity processing unit 13. The directivity processing unit 13 performs signal processing on the sound signals from the four microphones 12 a to 12 d to pick up only the sound from the specific direction. As signal processing, delay-type beamformers that enhance sound in a specific direction by shifting delays of sound signals from the four microphones 12a to 12d, or sound in a specific direction and sound in other unnecessary directions There is an adaptive beamformer method that generates by delay type and removes unwanted direction sound from sound in specific direction. Alternatively, there is a method of picking up sound in a specific direction with one unidirectional microphone. Alternatively, there is a method of preparing a plurality of unidirectional microphones for picking up sound in different directions, and selecting only the unidirectional microphone corresponding to a specific direction.

  When the “specific direction” is determined to have a certain degree of positional relationship between the wearer of the head mounted display 10 and an external person who talks to the wearer, for example, a passenger wearing the head mounted display 10 in an airplane In the case of limiting to the scene of and the cabin attendants, it is a predetermined fixed direction (for example, the direction toward the front of the aisle). However, the “specific direction” is not limited to the fixed direction, and may be a dynamically changing direction indicated from the outside.

  Further, the number of microphones constituting the sound collection unit 11 is not limited to four. For example, the sound collection unit 11 may be configured by three or more microphones as long as sound collection in a specific direction.

  The voice detection unit 14 is a processing unit that detects voice from the sound collected by the sound collection unit 11. For example, in the signal output from the sound collection unit 11, the power in the frequency band of the voice is a predetermined time, the threshold When it exceeds, the signal output from the sound collection unit 11 is determined as the voice, and the response detection unit 17 is notified that the voice is detected.

  The head movement detection unit 15 is a sensor that detects the movement of the head of the wearer of the head mounted display 10 and outputs a signal indicating the detected result to the response detection unit 17. For example, at least a gyro and an acceleration sensor It is one. The head movement detection unit 15 may detect the movement of the head by detecting the line of sight of the wearer using a camera or the like.

  In the response detection unit 17, the voice detection unit 14 detects voice and the head movement detection unit 15 detects motion such that the head of the wearer faces a specific direction in which the sound collection unit 11 picks up sound It is a processing unit that displays, on the external display 16, information indicating that the wearer has responded to the voice when it is received. In addition, "the head of a wearer turns to a specific direction" means "the face of a wearer turns to a specific direction".

  The internal display 34 is a display for presenting an image to a wearer of the head mounted display 10, and is, for example, a 3D image panel.

  The video reproduction unit 30 has a content storage unit 31 for storing content, reads out the content from the content storage unit 31 and reproduces it according to an instruction from the wearer by a remote control (not shown) or the like, and displays the video signal A processing unit that outputs a sound signal accompanying the video to the headphone 35 while outputting to 34, and is configured of, for example, a video decoder, a processor having a program, and the like.

  The directivity processing unit 13, the voice detection unit 14, and the response detection unit 17 are typically a ROM that holds a program, a RAM, a processor that executes the program, and an input / output circuit for connecting to peripheral circuits. And so on by software or a system circuit or a microcomputer. However, these may be realized in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements each process.

  Next, the operation of the head mounted display 10 according to the present embodiment configured as described above will be described.

  FIG. 3 is a flowchart showing an operation of the head mounted display 10 according to the present embodiment (that is, a control method of the head mounted display 10). Here, it is assumed that the head mounted display 10 is mounted while a passenger of an airplane is sitting in a chair and content is being reproduced.

  First, the sound collection unit 11 collects sound from a specific direction (sound collection step S10). Specifically, the directivity processing unit 13 performs signal processing such as a delayed beamformer on the sound signals from the four microphones 12a to 12d to make a specific direction (here, diagonally above). Pick up only the sound from the direction).

  Subsequently, the voice detection unit 14 detects voice from the sound collected by the sound collection unit 11 (voice detection step S11). Specifically, in the signal output from the sound collection unit 11, when the power in the frequency band of the sound exceeds the threshold for a certain period of time, the sound detection unit 14 generates the sound of the signal output from the sound collection unit 11 Then, the response detection unit 17 is notified that the voice has been detected. As a result, the response detection unit 17 determines whether the voice from diagonally above has been detected, that is, whether the cabin attendant spoke to the wearer of the head mounted display 10 or not.

  When the voice is detected by the voice detection unit 14 (YES in S11), the response detection unit 17 proceeds to the process after step S12, and otherwise ends the process (NO in S11).

  The head movement detection unit 15 detects the movement of the head of the wearer of the head mounted display 10, and outputs a signal indicating the detected result to the response detection unit 17 (head movement detection step S12).

  The response detection unit 17 analyzes the signal output from the head movement detection unit 15 to determine whether or not the wearer (here, the passenger) of the head mounted display 10 responds to a talk from the cabin attendant. To do (S13).

  Specifically, the response detection unit 17 detects that the head movement detection unit 15 detects that the wearer's head is directed in a specific direction in which the sound collection unit 11 picks up (S13 YES), it is determined that the wearer has responded to the call from the cabin attendant, and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

  After that, the response detection unit 17 determines whether the process termination condition is satisfied (S15), and continues to display information indicating that the wearer has responded to the voice on the external display 16 until the condition is satisfied. The “processing end condition” means that the wearer does not respond to the voice. For example, the head movement detection unit 15 causes the wearer's head to pick up the sound collection unit 11 so that the sound is collected. It is that the movement which turns to a direction is not detected.

  On the other hand, if the head movement detection unit 15 does not detect movement of the head of the wearer to face the specific direction in which the sound collection unit 11 picks up the response detection unit 17 (S13 NO), it is determined that the wearer did not respond to the call from the cabin attendant, and the process is ended.

  As described above, the head mounted display 10 according to the present embodiment is a head mounted display for presenting an image, and the sound collecting unit 11 for collecting sound from a specific direction and the sound collecting unit 11 A voice detection unit 14 for detecting a voice from the voiced sound, a head movement detection unit 15 for detecting the movement of the head of the wearer of the head mounted display 10, and an external display 16 for displaying information visible from outside When the voice detection unit 14 detects voice and the head movement detection unit 15 detects that the wearer's head is directed in a specific direction, the wearer responds to the voice And a response detection unit 17 for displaying the information to be displayed on the external display 16.

  In addition, the control method of the head mounted display 10 according to the present embodiment is a control method of the head mounted display 10 for presenting an image, and the sound collecting step S10 for collecting sound from a specific direction; Voice detection step S11 detects voice from the sound collected in step S10, head movement detection step S12 detects motion of the head of the wearer of the head mounted display 10, and voice detection step S11 detects voice And when the head movement detection step S12 detects that the wearer's head is oriented in a specific direction, information indicating that the wearer has responded to the voice can be viewed from the outside And a response detection step S14 displayed on the external display 16 for displaying information.

  This indicates that the wearer responds to the voice when it is detected that the face of the wearer of the head mounted display 10 faces a specific direction set as the sound source direction of the voice emitted by an external person. The information is displayed on the external display 16. Therefore, when the wearer indicates the intention to respond to the voice of the external person, that is displayed on the external display 16, so that the wearer and the external person can confirm the intentions of both parties. Smooth communication is possible.

  In the present embodiment, detection of head movement (S12) is performed after directivity processing (S10) and speech detection (S11) in the flowchart of FIG. 3, but the present invention is not limited to such an order. . The order of the directivity processing (S10) and the voice detection (S11) and the detection of head movement (S12) may be reversed, or may be performed in parallel.

(Modification 1)
Next, a head mounted display according to a first modification of the embodiment will be described.

  FIG. 4 is a block diagram showing the configuration of a head mounted display 10a according to a first modification of the embodiment. The head mounted display 10 a is a head mounted display having a function of identifying a speaker, and the sound collecting unit 11, the voice detecting unit 14, the speaker identifying unit 18, the head movement detecting unit 15, the external display 16, the response detecting unit 17a, a video reproduction unit 30, an internal display 34, and headphones 35.

  The head mounted display 10a according to the present modification has basically the same configuration as the head mounted display 10 according to the embodiment, but a point provided with the speaker identification unit 18 as a new component, and the modification This embodiment differs from the embodiment in that the response detection unit 17a according to the present embodiment is provided. Hereinafter, differences from the embodiment will be mainly described.

  The speaker identification unit 18 is a processing unit that determines whether the voice detected by the voice detection unit 14 is a voice of a predetermined person and notifies the response detection unit 17a of the determination result. In this modification, the speaker identification unit 18 has a storage unit in which the voices of the cabin attendants are registered in advance, and any of the voices detected by the voice detection unit 14 is registered in the storage unit. It is judged whether or not it matches.

  For example, the speaker identifying unit 18 extracts the feature amount of the speaker by using the i-vector and identifies the speaker (Non-Patent Document 1: Misaki Ayukawa et al., “Speaker identification of short utterance by i-vector Examination, “Shingaku Technical Report, vol. 115, no. 99, SP 2015-12, pp. 65-70, June 2015). That is, the speaker identification unit 18 converts the speech into multi-dimensional feature quantity vectors. The newly input speech is also converted into a feature quantity vector, and it is determined whether the input speech is the voice of a cabin attendant or not depending on whether it is close to any of the registered feature quantity vectors.

  The speaker identification unit 18 is typically a system circuit or a microcomputer including a ROM for holding a program, a RAM, a processor for executing the program, and an input / output circuit for connecting with a peripheral circuit. Is realized by software. However, these may be realized in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements each process.

  The response detection unit 17a basically has the same function as that of the response detection unit 17 according to the embodiment, and in this modification, it is assumed that the speaker identification unit 18 determines that the voice is a predetermined voice of a person. When the head movement detection unit 15 determines that the movement of the head of the wearer is directed in a specific direction, information indicating that the wearer has responded to the voice is displayed on the external display 16 Do.

  FIG. 5 is a flowchart showing an operation of the head mounted display 10a according to the present modification (that is, a control method of the head mounted display 10a). This operation is basically the same as the flowchart shown in FIG. 3 according to the embodiment, but differs from the embodiment in that step S20 is added. Hereinafter, differences from the embodiment will be mainly described.

  When the voice detected by the voice detection unit 14 is detected (YES in S11), the speaker identification unit 18 determines whether the voice detected by the voice detection unit 14 is a voice of a predetermined person. The determination is made, and the determination result is notified to the response detection unit 17a (S20). Specifically, the speaker identification unit 18 determines whether the feature quantity vector of the voice input via the voice detection unit 14 is close to any one of voice feature vectors of the cabin attendants registered in advance. And notifies the response detection unit 17a of the determination result.

  When the response detecting unit 17a is notified by the speaker identifying unit 18 that the input voice is a voice of a predetermined person (here, a cabin attendant) (YES in S20), The process proceeds to the process after S12, and if not (NO in S20), the process ends.

  In step S12 and subsequent steps, the response detection unit 17a detects that the head movement detection unit 15 detects that the wearer's head faces a specific direction in which the sound collection unit 11 picks up (S13 YES), it is determined that the wearer has responded to the call from the cabin attendant, and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14). It is similar.

  Thus, in addition to the configuration of the embodiment, the head mounted display 10a according to the present modification is a talk to determine whether or not the sound detected by the sound detection unit 14 is the sound of a predetermined person. A person identification unit 18 is provided. The response detecting unit 17a determines that the voice is a voice of a predetermined person by the speaker identifying unit 18 (S20), and the head movement detecting unit 15 turns the head of the wearer in a specific direction. When such a motion is detected (S13), information indicating that the wearer has responded to the voice is displayed on the external display 16.

  As a result, when the wearer of the head mounted display 10a indicates an intention to respond to a predetermined person's voice, that is displayed on the external display 16. Therefore, it is suppressed that the wearer erroneously responds to the voice from the irrelevant person as the noise jumping into the wearer, and the response accuracy of the head mounted display 10a is improved.

(Modification 2)
Next, a head mounted display according to a second modification of the embodiment will be described.

  FIG. 6 is a block diagram showing the configuration of a head mounted display 10b according to a second modification of the embodiment. The head mounted display 10b is a head mounted display having a function of searching for a sound source, and includes the sound collecting unit 11a, the sound detecting unit 14, the head movement detecting unit 15, the external display 16, the response detecting unit 17, the sound source searching unit 19, The video playback unit 30, the internal display 34, and the headphones 35 are provided.

  The head mounted display 10b according to the present modification basically has the same configuration as the head mounted display 10 according to the embodiment, but the point that the sound source search unit 19 is provided as a new component, and the modification This embodiment differs from the embodiment in that such a sound collection unit 11a is provided. Hereinafter, differences from the embodiment will be mainly described.

  The sound source search unit 19 is a processing unit that searches for a sound source. Specifically, the sound source search unit 19 searches for the direction or position (coordinates in a three-dimensional space) of the sound generated around using the four microphones 12a to 12d. For example, the sound source search unit 19 specifies the sound source direction in the plane including the two microphones based on the time difference that the same sound reaches each of the two microphones installed at a distance. Alternatively, the sound source search unit 19 specifies the intersection point of the two sound source directions as the position of the sound source by using two pairs of microphones arranged in different directions with the two microphones placed at a distance as a pair.

  The sound source search unit 19 is typically a system circuit or a microcomputer including a ROM holding a program, a RAM, a processor executing the program, and an input / output circuit for connecting to a peripheral circuit. , Is realized in software. However, these may be realized in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements each process.

  The sound pickup unit 11a picks up the direction of the sound source searched by the sound source search unit 19 as sound from a specific direction. Specifically, the directivity processing unit 13a configuring the sound collection unit 11a collects only the sound from the direction of the sound source searched by the sound source searching unit 19 with respect to the sound signals from the four microphones 12a to 12d. By performing signal processing for sound, only the sound from the direction of the sound source searched by the sound source search unit 19 is picked up. Signal processing is, for example, the same as the sound collection unit 11 of the embodiment.

  FIG. 7 is a flowchart showing an operation of the head mounted display 10b (that is, a control method of the head mounted display 10b) according to the present modification. This operation is basically the same as the flowchart shown in FIG. 3 according to the embodiment, but differs from the embodiment in that step S22 is added. Hereinafter, differences from the embodiment will be mainly described.

  The sound source search unit 19 searches for a sound source (S22) prior to the directivity processing (S10) by the directivity processing unit 13a. For example, the sound source search unit 19 searches for the position of the sound source (coordinates in a three-dimensional space) using the four microphones 12a to 12d. When the sound source is not searched (NO in S22), the process is ended.

  On the other hand, when the sound source is searched (YES in S22), the directivity processing unit 13a of the sound collection unit 11a collects the direction of the sound source searched by the sound source searching unit 19 as the sound from the specific direction. Sound (S10). Specifically, the directivity processing unit 13a performs signal processing for collecting only the sound from the direction of the sound source searched by the sound source searching unit 19 with respect to the sound signals from the four microphones 12a to 12d. By this, only the sound from the direction of the sound source searched by the sound source search unit 19 is picked up. The subsequent processing (S11) is the same as that of the embodiment.

  Thus, in addition to the configuration of the embodiment, the head mounted display 10b according to the present modification includes the sound source searching unit 19 that searches for a sound source. The sound pickup unit 11a picks up the direction of the sound source searched by the sound source search unit 19 as sound from a specific direction.

  As a result, since the direction of the sound source is searched by the sound source searching unit 19, it is not necessary to fix the voice of an external person to the voice from the specific direction. That is, when the wearer responds to the sound from all directions, that is displayed on the external display 16, and the convenience of the head mounted display 10b is improved.

(Modification 3)
Next, a head mounted display according to a third modification of the embodiment will be described.

  FIG. 8 is a block diagram showing the configuration of a head mounted display 10c according to a third modification of the embodiment. The head mounted display 10 c is a head mounted display having a function of superimposing a voice from an external person on the sound accompanying video reproduction and outputting it to the headphone 35, and the sound collecting unit 11 a, the sound detecting unit 14, the sound source searching unit 19 The audio superimposing unit 20, the head movement detecting unit 15, the external display 16, the response detecting unit 17, the video reproducing unit 30, the internal display 34, and the headphones 35 are provided.

  The head mounted display 10c according to the present modification basically has the same configuration as the head mounted display 10b according to the second modification of the embodiment, but in that the sound superimposing unit 20 is provided as a new component, It differs from the second modification of the embodiment. Hereinafter, differences from the second modification of the embodiment will be mainly described.

  The sound superimposing unit 20 is a processing unit that superimposes the sound detected by the sound detection unit 14 on the sound accompanying the video and outputs the sound to the headphone 35. At this time, the sound superimposing unit 20 performs sound image localization so that the sound detected by the sound detection unit 14 can be heard from the direction corresponding to the specific direction, and then superimposes the sound accompanying the video. Specifically, the sound superimposing unit 20 performs sound image localization so that the sound detected by the sound detection unit 14 can be three-dimensionally heard (that is, from the direction searched by the sound source searching unit 19). Signal processing, and then allocated to the reproduction channel of the headphone 35 and output. For example, the sound superimposing unit 20 performs signal processing of sound image localization on the sound detected by the sound detection unit 14 in consideration of the shape of a human head (that is, using a head-related transfer function). Thus, even if the headphones 35 are 2ch, the sound of an external person can be heard in three dimensions.

  In the present modification, the headphone 35 may be a closed type headphone because the user of the head mounted display 10 can hear the sound of an external person.

  In addition, the sound superimposing unit 20 is typically a system circuit or a microcomputer including a ROM for holding a program, a RAM, a processor for executing the program, and an input / output circuit for connecting to a peripheral circuit. , Is realized in software. However, these may be realized in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements each process.

  FIG. 9 is a flowchart showing an operation of the head mounted display 10 c (that is, a control method of the head mounted display 10 c) according to the present modification. This operation is basically the same as the flowchart shown in FIG. 7 according to the second modification of the embodiment, but differs from the second modification of the embodiment in that steps S24 and S26 are added. . Hereinafter, differences from the second modification of the embodiment will be mainly described.

  In the present modification, when a voice is detected by the voice detection unit 14 (S11), the voice superimposing unit 20 searches for the voice detected by the voice detection unit 14 by the sound source search unit 19 Signal processing for sound image localization is performed so as to be heard from the direction (S24), and the sound is superimposed on the sound reproduced by the image reproduction unit 30, and then output to the headphone 35 (S26).

  The subsequent processing (S13) is the same as that of the embodiment.

  Thus, in addition to the configuration of the embodiment, the head mounted display 10c according to the present modification superimposes the sound detected by the sound detection unit 14 on the sound accompanying the video and outputs the sound to the headphones 35. It has twenty.

  As a result, the voice of an external person (for example, a cabin attendant) is superimposed on the sound accompanying the video and output from the headphone 35, so the wearer of the head mounted display 10c views the voice of the content being reproduced. You can also hear the voices of people outside you. Therefore, more smooth communication is possible between the wearer and the outsider.

  Further, the sound superimposing unit 20 performs sound image localization so that the sound detected by the sound detection unit 14 can be heard from a direction corresponding to a specific direction, and then causes the sound to be superimposed on the sound accompanying the video.

  As a result, the wearer of the head mounted display 10c can hear the sound of an external person (for example, a cabin attendant) from the actual sound source direction or the corresponding sound source direction by sound image localization. As in the case of not wearing it, it is possible to easily detect the direction in which the external person who made the sound is present.

  In this modification, in the flowchart of FIG. 9, sound image localization superposition (S24) and reproduction (S26) are performed after head movement detection (S12), but the present invention is not limited to this order. The detection of the head movement (S12) and the sound image localization superposition (S24) and the reproduction (S26) may be switched in order or may be performed in parallel.

(Modification 4)
Next, as a fourth modification of the embodiment, an example in which the head mounted display according to the present disclosure is applied to a remote control system of a car will be described. The remote control system comprises a driver for remotely steering a vehicle, and a vehicle connected to the driver by wireless communication and remotely steered by the driver.

  FIG. 10A is an external view of a driving device 41 configuring a remote control system 40 according to the fourth modification of the embodiment. FIG. 10B is an external view of a car 45 constituting the remote control system 40 according to the fourth modification of the embodiment.

  As shown in FIG. 10A, the driving device 41 is a device installed indoors etc. and simulating the driver's seat of the automobile 45. The driving device 41 is configured of a steering wheel, an accelerator, a brake, and the like for remotely steering the automobile 45.

  The front side of the driving device 41 is provided with an equivalent to the internal display 34 of the head mounted display according to the embodiment (here, referred to as the internal display 34). In the present modification, the internal display 34 displays an image sent from a camera installed in the automobile 45 and imaging the front of the automobile 45. Further, the driving device 41 is provided with a speaker 36 corresponding to the headphones of the head mounted display according to the embodiment. In the present modification, the speaker 36 reproduces the sound collected by the microphones 12 a to 12 d attached to the front of the automobile 45 by the multi-speaker.

  As shown in FIG. 10B, the automobile 45 can be remotely steered by the driving device 41 and can travel unmanned. At four places on the front of the automobile 45, those corresponding to the microphones 12a to 12d of the head mounted display according to the embodiment (here, referred to as the microphones 12a to 12d) are attached. In addition, the front surface of the windshield of the automobile 45 is provided with one corresponding to the external display 16 of the head mounted display according to the embodiment (herein, referred to as the external display 16).

  FIG. 11 is a block diagram showing the main configuration of the remote control system 40 according to the present modification. Here, among the components of the remote control system 40, only the components where the head mounted display according to the embodiment is applied are shown.

  The remote control system 40 includes a sound collection unit 11 a, a sound detection unit 14, a sound source search unit 19, a sound superposition unit 20, a head movement detection unit 15, an external display 16, a response detection unit 17, a video reproduction unit 30, and an internal display 34. , And a speaker 36. This configuration corresponds to one in which the headphones 35 in the head mounted display 10c according to the third modification of the embodiment shown in FIG. However, in the present modification, the video reproduction unit 30 is the above-described camera installed in the automobile 45. Further, the voice detection unit 14 detects a vehicle surrounding sound registered in advance as a sound for predicting a danger such as a brake sound of the bicycle 46 shown in FIG. 10B. In addition, the head movement detection unit 15 is an acceleration sensor attached to the head of the remote pilot operated by the driving device 41 or a camera for capturing and detecting the movement of the head of the remote pilot. Also, typically, the sound collection unit 11a, the video reproduction unit 30, and the external display 16 are provided in the vehicle 45, and the other components are provided in the driving device 41 or attached to a remote operator. .

  As described above, since the remote control system 40 has the same configuration as the head mounted display 10c according to the third modification of the embodiment as a basic function, the same effect as the third modification of the embodiment is obtained. Played. That is, in the remote control system 40, by sound localizing the vehicle ambient sound such as the brake sound of the bicycle 46 detected by the sound detection unit 14, a sound for predicting danger can be heard from a direction close to the real sound source direction. A voice superimposing unit 20 that performs signal processing is provided. Therefore, when the head of the remote pilot faces the sound source direction of the vehicle ambient sound such as the brake sound of the bicycle 46, as shown in FIG. 10B, the fact that the remote pilot noticed the vehicle ambient sound is an automobile It is displayed on the 45 external displays 16.

  At this time, the external display 16 of the automobile 45 also looks at other than the driver of the bicycle 46. In order to intuitively indicate to the driver of the bicycle 46 that the display on the external display 16 is an indication indicating that the driver of the bicycle 46 has been noticed, display at a position close to the bicycle 46 or An arrow or an eyeball icon is directed in a direction, or a mirror image of a video taken by a camera of the bicycle 46 is displayed.

  As a result, the driver of the bicycle 46 can visually recognize that the vehicle 45 approaching him has noticed the brake sound of his / her bicycle 46. Therefore, according to the remote control system 40 to which such a head mounted display is applied, although the distance between the remote pilot and the car and its surroundings is large, the remote pilot and the external person (here, Smooth communication is possible with the driver of the bicycle 46).

(Modification 5)
Next, a head mounted display according to a fifth modification of the embodiment will be described.

  FIG. 12 is a block diagram showing the configuration of a head mounted display 10 d according to a fifth modification of the embodiment. The head mounted display 10d is a head mounted display having a function of automatically reducing the volume of the sound provided to the headphone, and the sound collecting unit 11, the voice detecting unit 14, the head movement detecting unit 15, the external display 16, the response detection A unit 17 b, a video reproduction unit 30, an internal display 34, headphones 35, and a sound amplifier 37 are provided.

  Although the head mounted display 10d according to the present modification has basically the same configuration as the head mounted display 10 according to the embodiment, the point provided with the sound amplifier 37 as a new component, and the modification This embodiment differs from the embodiment in that the response detection unit 17 b is provided. Hereinafter, differences from the embodiment will be mainly described.

  The sound amplifier 37 is a sound volume regulator that amplifies or attenuates the sound signal output from the video reproduction unit 30 in accordance with a control instruction from the response detection unit 17 b, and outputs the amplified signal to the headphone 35. Be done.

  The response detection unit 17b basically has the same function as the response detection unit 17 according to the embodiment, and in the present modification, further, information indicating that the wearer has responded to the voice is displayed on the external display 16 The sound amplifier 37 is controlled to lower the volume of the sound provided to the headphones 35 when displaying.

  FIG. 13 is a flowchart showing an operation of the head mounted display 10 d according to the present modification (that is, a control method of the head mounted display 10 d). This operation is basically the same as the flowchart shown in FIG. 3 according to the embodiment, but differs from the embodiment in that step S28 is added. Hereinafter, differences from the embodiment will be mainly described.

  When the head movement detection unit 15 detects that the head movement of the wearer is directed in a specific direction in which the sound collection unit 11 collects sound, the response detection unit 17 b detects headphones 35 (YES in S 13). The sound amplifier 37 is controlled to lower the volume of the sound to be provided (S28), and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

  Thus, in addition to the configuration of the embodiment, the head mounted display 10d according to the present modification includes the sound amplifier 37 that adjusts the volume of the sound output from the video reproduction unit 30. The response detection unit 17 b controls the sound amplifier 37 to lower the volume of the sound provided to the headphone 35 when the information indicating that the wearer has responded to the sound is displayed on the external display 16.

  This reduces the volume of the sound provided to the headphones 35 when an external person emits sound toward the wearer. Therefore, the wearer of the head mounted display 10 d can hear the voice of an external person while wearing the head mounted display 10 d.

(Modification 6)
Next, a head mounted display according to a sixth modification of the embodiment will be described.

  FIG. 14 is a block diagram showing a configuration of a head mounted display 10 e according to a sixth modification of the embodiment. The head mounted display 10 e is a head mounted display having a function of collecting not only the sound emitted by the wearer but also the external person, and the sound collecting unit 11 b, the sound detecting unit 14, the head movement detecting unit 15, and the external display 16. , A response detection unit 17c, a video reproduction unit 30, an internal display 34, and a headphone 35.

  The head mounted display 10e according to the present modification basically has the same configuration as the head mounted display 10 according to the embodiment, but in that the sound pickup unit 11b and the response detection unit 17c according to the modification are provided. It differs from the embodiment. Hereinafter, differences from the embodiment will be mainly described.

  The sound collection unit 11b basically has the same function as the sound collection unit 11 according to the embodiment, but in this modification, a directivity processing unit 13b according to a modification is included. Similar to the directivity processing unit 13 according to the embodiment, the directivity processing unit 13 b picks up only the sound from a specific direction by performing signal processing on the sound signals from the four microphones 12 a to 12 d. In the present modification, in accordance with the control instruction from the response detection unit 17c, the modification has a function of changing the direction in which the user picks up the sound emitted by the wearer. Specifically, the directivity processing unit 13b shifts the delays of the sound signals from the four microphones 12a to 12d as signal processing in accordance with the control instruction from the response detection unit 17c, thereby determining a predetermined specific direction. Or, a delayed beamformer or the like that emphasizes the sound from the direction of the wearer.

  The response detection unit 17c basically has the same function as the response detection unit 17 according to the embodiment, and in this modification, the voice detection unit 14 further detects voice and head movement detection. When the movement of the wearer's head is detected to a specific direction by the unit 15, the direction in which the sound collection unit 11b collects the sound is changed so that the sound emitted by the wearer is collected. Control.

  FIG. 15 is a flowchart showing an operation of the head mounted display 10 e according to the present modification (that is, a control method of the head mounted display 10 e). FIG. 16 is a view showing a display example on the external display 16 of the head mounted display 10 e according to the present modification.

  The operation shown in FIG. 15 is basically the same as the flowchart shown in FIG. 3 according to the embodiment, but the display (S30, S33, S35) on the external display 16 and the sound collection unit 11b are performed. This embodiment differs from the embodiment in that control (S31, S32) and the like are added. Hereinafter, differences from the embodiment will be mainly described.

  When a voice is detected by the voice detection unit 14 (YES in S11), the response detection unit 17c displays an image indicating that the voice detection unit 14 detects a voice as shown in the display example of (b) in FIG. That is, when the cabin attendant is calling on the wearer of the head mounted display 10 e, an image showing that the microphones 12 a to 12 d are functioning (here, an image showing the microphone etc.) is displayed on the external display 16 (S30). Thereby, the external display 16 displays an image indicating that the microphones 12a to 12d are functioning from the normal display state (here, no display shown in FIG. 16A). Change to (b). Thus, the cabin attendant can know that his / her request has been detected by the microphones 12a to 12d of the head mounted display 10e.

  Subsequently, when movement is detected such that the head of the wearer points in a specific direction in which the sound collection unit 11b picks up (YES in S13), the response detection unit 17c collects the voice emitted by the wearer. Control is performed to change the direction in which the sound collection unit 11b picks up sound so as to sound (S31). As a result, the sound collection unit 11b performs directivity processing to collect the voice of the wearer (S32).

  Then, the response detection unit 17c displays an image (here, a question mark) indicating that the wearer has responded on the external display 16 as in the display example of (c) in FIG. 16 (S33). As a result, the cabin attendant can know that the wearer of the head mounted display 10 e notices to his / her request.

  After that, the response detection unit 17c determines whether the process termination condition is satisfied (S15). If not satisfied (NO in S15), the response detection unit 17c is attached via the sound collection unit 11b and the voice detection unit 14 It is determined whether the voice of the person is detected (S34), and if the voice of the wearer is not detected (NO in S34), the response display (S33) and the termination determination (S15) are repeated.

  On the other hand, when the voice of the wearer is detected (YES in S34), the response detection unit 17c indicates that the wearer has uttered as shown in the display example of (d) of FIG. An image (here, an exclamation mark) indicating that the cabin attendant has responded to the call is displayed on the external display 16 (S33). As a result, the cabin attendant can know that the wearer of the head mounted display 10 e responds to his call.

  After the response detection unit 17c performs the display shown in (d) of FIG. 16, it is considered that a series of conversations in which the wearer of the head mounted display 10e returns a response to the call from the cabin attendant is finished. As shown in the display example of (e) of FIG. 16, the external display 16 is returned to the normal display state as in (a) of FIG.

  As described above, in the head mounted display 10 e according to the present modification, the voice detection unit 14 detects a voice in the response detection unit 17 c, and the head movement detection unit 15 detects the head of the wearer in a specific direction. When a movement to turn is detected, control is performed to change the direction in which the sound collection unit 11b collects sound so as to collect the sound emitted by the wearer.

  As a result, after the wearer responds to the voice from the external person, the sound collection direction is changed so as to pick up the voice from the wearer, so that the voice emitted by the wearer is It is picked up. Therefore, more smooth communication is possible between the wearer and the outsider.

(Modification 7)
Next, a head mounted display according to Modification 7 of the embodiment will be described.

  FIG. 17 is a block diagram showing the configuration of a head mounted display 10 f according to the seventh modification of the embodiment. The head mounted display 10 f is a head mounted display that enables a wearer to smoothly communicate with an external person while confirming the intention of both with an external person while wearing the head mounted display 10 f. The voice detection unit 14, the body motion detection unit 15A, the external display 16, the response detection unit 17, the video reproduction unit 30, the internal display 34, and the headphones 35 are provided.

  The head mounted display 10f according to this modification basically has the same configuration as the head mounted display 10 according to the embodiment, but instead of the head movement detecting unit 15, body movement detection as a new component This embodiment differs from the embodiment in that the unit 15A is provided. Hereinafter, differences from the embodiment will be mainly described.

  The body motion detection unit 15A detects the direction of the body and determines the response to the voice. As a means for detecting the direction of the body, a means for recognizing an image using an image of a camera directed to the wearer of the head mounted display 10 f or an acceleration sensor or a gyro sensor is mounted on the body of the wearer of the head mounted display 10 f There is a means. In addition, when the remote control is held by the hand, an acceleration sensor or a gyro sensor mounted in the remote control can be used. It is determined by the body motion detection unit 15A that the body has responded to the voice when the body has turned in a specific direction.

  FIG. 18 is a flowchart showing an operation of the head mounted display 10 f (that is, a control method of the head mounted display 10 f) according to the present modification. This operation is basically the same as the flowchart shown in FIG. 3 according to the embodiment, but differs from the embodiment in that step S12 is replaced with S12A. Hereinafter, differences from the embodiment will be mainly described.

  When the voice is detected by the voice detection unit 14 (YES in S11), the response detection unit 17 proceeds to the process after step S12A, and otherwise ends the process (NO in S11).

  The body movement detection unit 15A detects the movement of the body of the wearer of the head mounted display 10f, and outputs a signal indicating the detected result to the response detection unit 17 (body movement detection step S12A).

  The response detection unit 17 analyzes the signal output from the body movement detection unit 15A to determine whether the wearer (here, the passenger) of the head mounted display 10 has responded to the talk from the cabin attendant. (S13).

  Specifically, the response detecting unit 17 detects that the body motion detecting unit 15A has turned the wearer's body in a specific direction in a specific direction in which the sound collecting unit 11 picks up (S13 YES), it is determined that the wearer has responded to the talk from the cabin attendant, and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

  After that, the response detection unit 17 determines whether the process termination condition is satisfied (S15), and continues to display information indicating that the wearer has responded to the voice on the external display 16 until the condition is satisfied. The “processing end condition” means that the wearer does not respond to the voice. For example, the body motion detection unit 15A allows the body of the wearer to pick up a specific direction in which the sound collection unit 11 collects sound. It is that the movement which turns is not detected.

  On the other hand, if the body motion detection unit 15A does not detect that the body of the wearer turns to the specific direction in which the sound collection unit 11 picks up the response detection unit 17 (NO in S13) It is determined that the wearer did not respond to the call from the cabin attendant, and the process is ended.

  As described above, the head mounted display 10 f according to the present embodiment is a head mounted display for presenting an image, and the sound collecting unit 11 for collecting sound from a specific direction, and the sound collecting unit 11 A voice detection unit 14 for detecting a voice from the voiced sound, a body motion detection unit 15A for detecting the orientation of the body of the wearer of the head mounted display 10, an external display 16 for displaying information visible from outside, voice When the detection unit 14 detects a voice and when the body motion detection unit 15A detects that the body of the wearer turns in a specific direction, the external display shows information indicating that the wearer responded to the voice. And a response detection unit 17 to be displayed at 16.

  In addition, the control method of the head mounted display 10 according to the present embodiment is a control method of the head mounted display 10 for presenting an image, and the sound collecting step S10 for collecting sound from a specific direction; A voice is detected by a voice detection step S11 for detecting voice from the sound collected in step S10, a body motion detection step S12A for sensing the orientation of the wearer of the head mounted display 10, and a voice detection step S11. When the body motion detection step S12A detects that the body of the wearer has turned to the specific direction, information indicating that the wearer has responded to the voice is displayed as visible information from the outside. And a response detection step S14 to be displayed on the external display 16.

  This indicates that the wearer responds to the voice when it is detected that the body of the wearer of the head mounted display 10 faces a specific direction set to the sound source direction of the voice emitted by an external person. The information is displayed on the external display 16. Therefore, when the wearer indicates the intention to respond to the voice of the external person, that is displayed on the external display 16, so that the wearer and the external person can confirm the intentions of both parties. Smooth communication is possible.

  In the present embodiment, detection of body movement (S12A) is performed after directivity processing (S10) and speech detection (S11) in the flowchart of FIG. 18, but the present invention is not limited to such an order. The order of the directivity processing (S10) and the voice detection (S11) and the detection of body movement (S12A) may be reversed, or may be performed in parallel.

(Modification 8)
Next, a head mounted display according to Modification 8 of the embodiment will be described.

  FIG. 19 is a block diagram showing a configuration of a head mounted display 10g according to a modified example 8 of the embodiment. The head mounted display 10 g is a head mounted display that enables a wearer to smoothly communicate with an external person while confirming the intention of both with the head mounted display 10 g. The voice detection unit 14, the hand and arm operation detection unit 15B, the external display 16, the response detection unit 17, the video reproduction unit 30, the internal display 34, and the headphones 35 are provided.

  The head mounted display 10g according to the present modification has basically the same configuration as the head mounted display 10 according to the embodiment, but instead of the head movement detecting unit 15, hand and arm movement detection as a new component This embodiment differs from the embodiment in that the unit 15B is provided. Hereinafter, differences from the embodiment will be mainly described.

  The hand and arm movement detection unit 15B detects the movement of the hand or arm and determines the response to the voice. As a means for detecting the movement of the hand or arm, a means for recognizing an image using an image of a camera directed to the wearer of the head mounted display 10g, an acceleration sensor or a gyro sensor on the body of the wearer of the head mounted display 10g There is a means to wear. In addition, when the remote control is held by the hand, an acceleration sensor or a gyro sensor mounted in the remote control can be used. It is determined that the user has responded to the voice by performing a predetermined movement of the hand or arm (for example, raising the hand up) by the hand and arm movement detection unit 15B.

  FIG. 20 is a flowchart showing an operation of the head mounted display 10 g according to the present modification (that is, a control method of the head mounted display 10 g). This operation is basically the same as the flowchart shown in FIG. 3 according to the embodiment, but differs from the embodiment in that step S12 is replaced with S12B. Hereinafter, differences from the embodiment will be mainly described.

  When the voice is detected by the voice detection unit 14 (YES in S11), the response detection unit 17 proceeds to the process after step S12B, otherwise (NO in S11), the process ends.

  The hand and arm movement detection unit 15B detects the movement of the wearer's hand and arm of the head mounted display 10g, and outputs a signal indicating the detected result to the response detection unit 17 (hand and arm movement detection step S12B).

  The response detection unit 17 analyzes the signal output from the hand and arm movement detection unit 15B to determine whether or not the wearer (here, the passenger) of the head mounted display 10 responds to a talk from the cabin attendant. (S13).

  Specifically, when the hand and arm movement detection unit 15B detects a predetermined movement of the wearer's hand or arm in a specific direction in which the sound collection unit 11 picks up the response detection unit 17 (YES in S13 ), It is determined that the wearer has responded to the call from the cabin attendant, and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

  After that, the response detection unit 17 determines whether the process termination condition is satisfied (S15), and continues to display information indicating that the wearer has responded to the voice on the external display 16 until the condition is satisfied. The “processing end condition” is that the wearer does not respond to the voice, and for example, the hand and arm movement detection unit 15B does not detect a predetermined movement of the wearer's hand or arm.

  On the other hand, when the predetermined movement of the wearer's hand or arm is not detected by the hand and arm movement detection unit 15B (NO in S13), the response detection unit 17 responds to the talk from the cabin attendant. It is determined that there is no, and the process is ended.

  As described above, the head mounted display 10 g according to the present embodiment is a head mounted display for presenting an image, and the sound collecting unit 11 for collecting sound from a specific direction, and the sound collecting unit 11 A voice detection unit 14 for detecting a voice from the voiced sound, a hand and arm operation detection unit 15B for detecting the movement of the hand or arm of the wearer of the head mounted display 10, an external display 16 for displaying information visible from outside The voice is detected by the voice detection unit 14, and when the predetermined movement of the wearer's hand or arm is detected by the hand and arm motion detection unit 15B, the external display indicates that the wearer responds to the voice And a response detection unit 17 to be displayed at 16.

  In addition, the control method of the head mounted display 10 according to the present embodiment is a control method of the head mounted display 10 for presenting an image, and the sound collecting step S10 for collecting sound from a specific direction; Voice is detected by voice detection step S11 which detects voice from the sound collected in step S10, hand and arm operation detection step S12B which detects the movement of the hand or arm of the wearer of the head mounted display 10, voice detection step S11. When the predetermined movement of the wearer's hand or arm is detected by the hand and arm movement detection step S12B, information indicating that the wearer has responded to the voice is displayed as information that can be viewed from the outside And a response detection step S14 displayed on the display 16.

  This indicates that the wearer responds to the voice when it is detected that the body of the wearer of the head mounted display 10 faces a specific direction set to the sound source direction of the voice emitted by an external person. The information is displayed on the external display 16. Therefore, when the wearer indicates the intention to respond to the voice of the external person, that is displayed on the external display 16, so that the wearer and the external person can confirm the intentions of both parties. Smooth communication is possible.

  In the present embodiment, detection of hand and arm movement (S12B) is performed after directivity processing (S10) and voice detection (S11) in the flowchart of FIG. 20, but the present invention is not limited to such an order. The order of the directivity processing (S10), the voice detection (S11), and the detection of hand and arm movement (S12B) may be reversed, or may be performed in parallel.

  As mentioned above, although the head mounted display concerning this indication and its control method were explained based on an embodiment and a modification, this indication is not limited to these embodiments and a modification. As long as the gist of the present disclosure is not deviated, various modifications that can be conceived by those skilled in the art are applied to the embodiment and the modification, and another embodiment constructed by combining some components in the embodiment and the modification is also possible. , Included within the scope of the present disclosure.

  For example, the sound source search unit 19 according to the second modification, the sound superimposing unit 20 according to the third modification, and the sound amplifier 37 according to the fifth modification are added to the head mounted display 10a according to the first modification of the embodiment. The response detection unit may have the function according to these modifications. As a result, a highly functional head mounted display that achieves the effects according to each modification is realized.

  Further, in the embodiment and the modifications, each component may be configured by dedicated hardware or implemented by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, software for realizing the head mounted display and the like according to the embodiment and the modification is a program as follows.

  That is, this program causes a computer to execute a control method of a head mounted display that presents an image. The control method is a control method of the head mounted display 10 that presents an image, and detects an audio from the sound collected in the sound collection step S10 and the sound collection step S10 in which the sound from the specific direction is collected. A voice is detected in the following voice detection step S11, a head movement detection step S12 for detecting the movement of the head of the wearer of the head mounted display 10, a voice detection step S11, and mounting in the head movement detection step S12. Response detection that displays information indicating that the wearer has responded to the voice on the external display 16 that displays information that can be viewed from the outside, when a movement is detected such that the head of the person turns in a specific direction And step S14.

  In addition, this program changes the head movement detection step S12 for detecting the movement of the head of the wearer of the head mounted display 10, and the body movement detection step S12A for detecting the orientation of the body of the wearer of the head mounted display 10f. Alternatively, a hand and arm movement detection step S12B may be used to detect a predetermined movement of the hand or arm of the head mounted display 10g.

  The present invention relates to a head mounted display and a remote control system, and more particularly, to a head mounted display that enables a wearer of the head mounted display and an external person to confirm and agree with each other. It can be used as a remote control system.

Reference Signs List 10, 10a to 10g head mounted display 11, 11a, 11b sound collecting unit 12a to 12d microphone 13, 13a, 13b directivity processing unit 14 voice detection unit 15 head movement detection unit 15A physical movement detection unit 15B hand arm movement detection unit 16 External display 17, 17a to 17c Response detection unit 18 Speaker identification unit 19 Sound source search unit 20 Voice superposition unit 30 Image reproduction unit 31 Content storage unit 34 Internal display 35 Headphones 36 Speaker 37 Sound amplifier 40 Remote control system 41 Driving device 45 Automobile 46 bicycle

Claims (13)

A head mounted display that presents an image,
A sound pickup unit that picks up sound from a specific direction,
A voice detection unit that detects voice from the sound collected by the sound collection unit;
A head movement detection unit for detecting movement of a head of a wearer of the head mounted display;
An external display that displays externally visible information;
The wearer responds to the voice when the voice detection unit detects a voice and the head movement detection unit detects a motion such that the head of the wearer points in the specific direction. And a response detection unit that displays information indicating that an event has occurred on the external display. The voice recognition unit further includes a speaker identification unit that determines whether the voice detected by the voice detection unit is a voice of a predetermined person.
The response detection unit is determined by the speaker identification unit that the voice is a voice of a predetermined person, and the head movement detection unit directs the head of the wearer to the specific direction. The head mounted display according to claim 1, wherein the information is displayed on the external display when such movement is detected. Furthermore, a sound source search unit for searching for a sound source is provided,
The head mounted display according to claim 1 or 2, wherein the sound collection unit picks up the direction of the sound source searched by the sound source searching unit as a sound from the specific direction. Furthermore, headphones providing the wearer with the sound accompanying the video;
The head mounted display according to any one of claims 1 to 3, further comprising: an audio superimposing unit that superimposes the sound detected by the sound detection unit on the sound accompanying the video and outputs the sound to the headphone. The head according to claim 4, wherein the sound superimposing unit performs sound image localization so that the sound detected by the sound detecting unit can be heard from a direction corresponding to the specific direction, and then superimposes the sound accompanying the image. Mount display. And a headphone for providing the wearer with a sound accompanying the image.
The head mounted display according to any one of claims 1 to 3, wherein the response detection unit reduces the volume of the sound provided to the headphone when the information is displayed on the external display. The said response detection part is the said mounting | wearing, when a voice is detected by the said audio | voice detection part, and the motion where the said wearer's head faces the said specific direction is detected by the said head movement detection part. The head mounted display according to any one of claims 1 to 6, wherein control is performed to change a direction in which the sound collection unit collects sound so as to collect voice emitted by a person. A control method of a head mounted display for presenting an image, comprising:
A sound collection step for collecting sound from a specific direction,
A voice detection step of detecting voice from the sound picked up in the sound collection step;
A head movement detection step of detecting movement of a head of a wearer of the head mounted display;
The wearer responds to the voice when the voice detection step detects a voice and the head motion detection step detects a motion such that the head of the wearer points in the specific direction. A response detection step of displaying information indicating that the information has been displayed on an external display that displays information that can be visually recognized from the outside.   A program that causes a computer to execute the steps included in the control method according to claim 8. A head mounted display that presents an image,
A sound pickup unit that picks up sound from a specific direction,
A voice detection unit that detects voice from the sound collected by the sound collection unit;
A body motion detection unit that detects a body orientation of a wearer of the head mounted display;
An external display that displays externally visible information;
When the voice detection unit detects a voice and the body motion detection unit detects that the body of the wearer has turned in the specific direction, the wearer responds to the voice A response detection unit that displays information to be displayed on the external display. A control method of a head mounted display for presenting an image, comprising:
A sound collection step for collecting sound from a specific direction,
A voice detection step of detecting voice from the sound picked up in the sound collection step;
A body motion detection step of detecting a body orientation of a wearer of the head mounted display;
When the voice detection step detects voice and the body motion detection step detects that the body of the wearer has turned to the particular direction, the wearer responds to the voice And D. a response detection step of displaying the information to be displayed on an external display that displays externally visible information. A head mounted display that presents an image,
A sound pickup unit that picks up sound from a specific direction,
A voice detection unit that detects voice from the sound collected by the sound collection unit;
A hand and arm movement detection unit that detects movement of a hand or arm of a wearer of the head mounted display;
An external display that displays externally visible information;
Information indicating that the wearer responds to the voice when the voice detection unit detects voice and the hand and arm motion detection unit detects a predetermined movement of the wearer's hand or arm A head mounted display comprising: a response detection unit for displaying on the external display. A control method of a head mounted display for presenting an image, comprising:
A sound collection step for collecting sound from a specific direction,
A voice detection step of detecting voice from the sound picked up in the sound collection step;
A hand and arm movement detecting step of detecting movement of a hand or arm of a wearer of the head mounted display;
When the voice detection step detects voice and the hand and arm motion detection unit detects a predetermined movement of the wearer's hand or arm, information indicating that the wearer responds to the voice is used. And a response detection step of displaying information visible from the outside on an external display.

Images