JP2019033426A - Video sound reproduction device and method - Google Patents

Abstract

To provide a video sound reproduction device which allow for viewing and listening contents of both VR and AR.SOLUTION: A video sound reproduction device includes a reproduction circuit capable of reproducing a virtual reality content and an augmented reality content, a control circuit for controlling the reproduction circuit to reproduce any one of the virtual reality content and the augmented reality content selectively, a first display for displaying the image of the virtual reality content, a second display for displaying the image of the augmented reality content, and a head mount mechanism capable of replacing the first and second displays.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video / audio reproduction device, and more particularly, to a video / audio reproduction device using a head-mounted display.

  Virtual reality (hereinafter referred to as “VR”) is a game, travel, sport, and so on using a goggle-type display called a head-mounted display (HMD) that displays 360-degree video, an eyepiece, and 3D sound technology. It is a technology for reproducing content such as a movie as if it were real, and it can provide a dramatic immersion compared to conventional flat-screen TV images.

  Augmented Reality (Augmented Reality, hereinafter referred to as “AR”) is a technology that combines the real world and virtual reality, and is content that can be used outdoors, for example, as portable game software. It is expected to become a big market. AR is also called Mixed Reality, which is created by real images taken by the camera and computer graphics by placing an environment recognition camera and a space recognition camera in the head mounted display housing. Holographic optical technology has also been put into practical use in which video is synthesized in real time and displayed on a display again with a micro display.

  In International Publication No. 2014/115393 “Image Display Device and Image Display Method and Image Display System” published on July 31, 2014, the image display device is a light-shielding type used for VR and a transmission type used for AR. Can understand the outline of HMD.

  An example of the details of VR can be referred to in its entirety in US Patent Application Publication No. 2016/03361658, “Expanded Field of View re-rendering for VR Spectral” published on December 15, 2016. In this invention, when a user wearing an HMD executes a video game, the user's operation is detected by the camera so that the operation does not cause a delay between video and audio, and a video server on the network is newly added. Is used to generate and provide video with a wider field of view (FOV).

  An example of details of AR can be referred to in its entirety in US Patent Application Publication No. 2014/0168261 “Direct interaction system mixed reality environments” filed on December 13, 2012. In accordance with the present invention, a system configuration for interacting with virtual objects in a mixed reality environment tracks a 3D scene map generated by a head mounted display device (HMD) that includes a camera that captures the user's field of view, The mobile processing unit can display virtual objects in the real world within the user's field of view (FOV). The transparency of the HMD provides a function of projecting a virtual image in the user's field of view so that the virtual image is displayed next to the real object. It also automatically tracks where the user is looking and allows the system to determine where to insert the virtual image into the user's FOV. When the system knows where to project the virtual image, the image is projected using the display element.

International Publication No. 2014/115393 US Patent Application Publication No. 2016/0361658 US Patent Application Publication No. 2014/0168261

  However, VR is used mainly indoors from the viewpoint of safety because human vision is blocked by a display in order to enjoy 360-degree video. On the other hand, the AR employs a transmissive display and is safe, but it is not suitable for viewing a 360-degree dynamic image because the viewing angle is limited. The reason is that VR requires a high immersive feeling of FOV 100 degrees or more, whereas AR is used for PC applications such as character recognition even if FOV is about 50 degrees and the field of view is narrow to combine virtual reality with the real world. In many cases, high resolution is required. The number of pixels per degree is used as a resolution evaluation standard. In the case of VR, since the viewing angle is about 110 degrees, the horizontal resolution is about 20 pixels / degree (2160 pixels) when 2K organic EL is used. / 110 degrees = 19.6), and in the case of AR or MR, since the viewing angle is about 50 degrees, the horizontal resolution is about 43 pixels / degree (2160 pixels / 50 degrees = 43.2) in the case of 2K. It can be seen that the resolution of AR is high. In other words, since the use is different, the optical specifications and configuration of the display to be used are different between VR and AR.

  At this stage, it is necessary to improve the resolution of the image in order to get closer to the real world, but high-resolution organic EL displays (for example, 8K and 12K panels) are not mass-productive and have high manufacturing costs. In addition, when detecting the movement of the user's head and playing a video on a high-resolution panel, if a video time delay occurs, a phenomenon called VR sickness occurs, which hinders long-time viewing. For this reason, a signal processing capability with high performance is required to realize a high frame rate of 90 Hz or higher, but there is a big problem for the portable terminal that the load of the signal processing semiconductor is large and the power consumption increases.

  Since VR and AR also have different content to be played back, it is burdensome for the user to purchase two VR-specific head-mounted display devices with priority on viewing angle and two AR-specific head-mounted display devices with priority on resolution. A composite display that combines VR and AR technology has also been studied, but has not yet been put into practical use. Even if such a composite display is realized, the weight is likely to increase, and fatigue due to wearing for a long time can be considered.

  In view of the above problems, an object of the present invention is to provide a video / audio reproduction device capable of viewing both VR and AR content.

In order to achieve the above object, the video / audio reproduction device of the present invention includes:
A playback circuit unit capable of playing back virtual reality content and augmented reality content;
A control circuit unit that selects and reproduces either the virtual reality content or the augmented reality content on the reproduction circuit unit;
A first display unit displaying an image of virtual reality content;
A second display unit displaying an image of augmented reality content;
A head mount mechanism capable of detaching and replacing the first display unit and the second display unit;

  According to the configuration of the present invention, it is possible to provide a video / audio reproduction device capable of viewing both VR and AR contents.

Schematic diagram of head mounted display (HMD) according to the first embodiment Side view of appearance configuration of HMD when reproducing VR content of first embodiment Side view of appearance configuration of HMD when reproducing AR content of one embodiment The block diagram which shows the functional component of HMD of 1st embodiment Schematic diagram showing an example of a mechanism for attaching and detaching the AR unit to the HMD device Side view of the HMD device of the second embodiment Block diagram of the second embodiment Conceptual diagram for detecting the position of the earphone unit The graph showing the relationship between the distance between the beacon terminal built in the earphone and the signal detector and the radio wave intensity RSSI Schematic diagram showing an example of displaying the distance from the beacon signal source in text on the display Schematic diagram showing an example of graphically displaying the direction of the beacon signal source on the display Side view of the HMD of the third embodiment Block diagram of the third embodiment

The video and audio playback device according to the first configuration of the present invention is as follows.
A playback circuit unit capable of playing back virtual reality content and augmented reality content;
A control circuit unit that selects and reproduces either the virtual reality content or the augmented reality content on the reproduction circuit unit;
A first display unit displaying an image of virtual reality content;
A second display unit displaying an image of augmented reality content;
A head mount mechanism capable of detaching and replacing the first display unit and the second display unit;

  According to this configuration, by exchanging the first display unit that displays the image of the virtual reality content and the second display unit that displays the image of the augmented reality content according to the content to be reproduced, It is possible to view both VR and AR content with a single device.

The video and audio playback device according to the second configuration further includes:
A detection means for detecting which of the first display unit and the second display unit is attached to the head mount mechanism;
When it is detected that the first display unit is mounted, the control circuit unit displays a selection screen of virtual reality content that can be played back by the playback circuit unit on the first display unit,
When it is detected that the second display unit is attached, the control circuit unit displays an augmented reality content selection screen that can be played back by the playback circuit unit on the second display unit.

  According to this configuration, the detection unit automatically detects which of the first display unit and the second display unit is mounted on the head mount mechanism, and according to the mounted display unit, A screen for selecting VR content and AR content can be displayed. This saves the user the trouble of selecting which VR content or AR content is to be played.

The video / audio reproduction device according to the third configuration may further include the first or second configuration,
Further comprising a sound reproducing device that can be deployed in the vicinity of at least one of the left and right auricles or the ear canal,
The control circuit unit is provided in the sound reproducing device.

  According to this configuration, the head mount mechanism can be reduced in weight by providing the control circuit unit in the sound reproducing device. As a result, it is possible to provide a video and audio playback device that is less fatigued even when viewed for a long time.

The video and audio reproduction device according to the fourth configuration further includes the third configuration,
The sound reproduction device includes an input device for inputting an operation instruction to the control circuit unit.

  According to this configuration, the input device is provided in the sound reproducing device that can be arranged in the vicinity of at least one of the left and right auricles or in the external auditory canal, so that, for example, the user operates by holding a separate remote controller. Compared to the case, the degree of freedom of both hands is improved.

The video and audio reproduction device according to the fifth configuration further includes the third or fourth configuration,
The sound reproduction device further includes a position transmission unit that transmits a position signal indicating a current position of the sound reproduction device,
The video and audio reproduction device further includes a position detection unit that detects a position signal transmitted from the position transmission unit,
The control circuit unit displays the position of the sound reproducing device on the first display unit or the second display unit based on the position signal detected by the position detection unit.

  According to this configuration, when the sound reproduction device is lost, the sound reproduction device can be detected based on the position signal transmitted from the sound reproduction device, and the position can be displayed on the display unit. Thereby, even if the sound reproducing device is lost, it can be easily found.

The video and audio reproduction device according to the sixth configuration is the fifth configuration,
The position transmission unit transmits a beacon signal as the position signal.

The video and audio reproduction device according to the seventh configuration is the sixth configuration,
The position detection unit is provided on both the left and right sides of the video and audio playback device.

The video and audio playback device according to the eighth configuration further includes:
The distance to the sound reproduction device detected by the position detection unit provided on the left side of the video / audio reproduction device and the sound reproduction device detected by the position detection unit provided on the right side of the video / audio reproduction device The control circuit unit detects the position of the sound reproducing device using the difference from the distance.

  According to this configuration, based on the difference between the distance to the sound reproduction device detected on the left side of the video / audio reproduction device and the distance to the sound reproduction device detected on the right side of the video / audio reproduction device. Then, it is possible to obtain a certain direction of the sound reproducing device and a distance to the direction.

The audiovisual reproduction apparatus according to the ninth configuration is any one of the sixth to eighth configurations,
The control circuit unit causes the first display unit or the second display unit to display the distance to the position transmission unit that is the transmission source of the position signal in text.

  Thus, by displaying the position of the sound reproduction device in text, it is possible to easily find the sound reproduction device.

The audiovisual reproduction apparatus according to the tenth configuration is any one of the sixth to ninth configurations,
The control circuit unit causes the first display unit or the second display unit to graphically display the direction of the position transmission unit that is a transmission source of the position signal.

  Thus, by displaying the position of the sound reproduction device in text, it is possible to easily find the sound reproduction device.

The video and audio reproduction device according to the eleventh configuration is any one of the first to tenth configurations,
A first microphone capable of detecting an external noise signal and a second microphone capable of detecting an external audio signal are further provided.

Furthermore, the video and audio playback device according to the twelfth configuration is further configured according to the eleventh configuration.
When the playback circuit unit is playing back virtual reality content, the first microphone operates preferentially,
When the playback circuit unit is playing back augmented reality content, the second microphone operates preferentially.

  According to this configuration, when the VR content is being reproduced, an external noise signal is detected by the first microphone and, for example, a noise canceling process is performed, so that a situation in which the VR content is easily immersed can be created. Further, when reproducing AR content, it is possible to capture real-world sound by detecting an external audio signal with the second microphone.

The sound reproducing device according to the thirteenth configuration of the present invention is
An audio reproduction device that is used together with a video and audio reproduction device including a reproduction circuit unit capable of reproducing virtual reality content and augmented reality content, and that can be deployed in the vicinity of at least one of the left and right auricles or the ear canal,
A control circuit unit is provided that selects any one of the virtual reality content and the augmented reality content and causes the reproduction circuit unit to reproduce the content.

  In this configuration, the reproduction circuit unit is provided in a sound reproduction device that can be disposed in the vicinity of at least one of the left and right auricles or in the ear canal. Accordingly, the audio / video reproduction apparatus can be controlled by the audio reproduction apparatus.

The sound reproducing device according to the fourteenth configuration is the thirteenth configuration,
A control circuit and a sensor used for operation of the video / audio reproduction device are provided.

  According to this configuration, the head mounting mechanism can be reduced in weight by providing the control circuit unit and the sensor in the sound reproducing device. As a result, it is possible to provide a video and audio playback device that is less fatigued even when viewed for a long time.

  Hereinafter, more specific embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a conceptual diagram of a head mounted display (HMD) according to an embodiment of the present invention, and shows that an AR unit and a VR unit can be exchanged. FIG. 2 is a side view of the HMD device when the VR unit according to the first embodiment is mounted. FIG. 3 is a side view of the HMD device when the AR unit of the first embodiment is mounted. FIG. 4 is a block diagram showing functional components of the HMD of the first embodiment.

  FIG. 1 shows the concept of the HMD of this embodiment. The VR unit 11 and the AR unit 12 are interchangeable, and are joined to each other by the AR / VR common unit 10 and the unit engaging portions 14a and 14b. The VR unit 11 and the AR unit 12 are mainly configured by an optical module, and the AR / VR common unit 10 includes a signal processing circuit, a battery, an attitude detection sensor, and the like. The unit engaging portions 14a and 14b are electrically coupled by a connector.

  The user wears the VR unit 11 when viewing immersive content (VR content) such as a movie, a sport, a concert, or a competitive game. When viewing AR content for collaborative work with a plurality of people, such as product development or in-house education, in which a virtual image is displayed in three dimensions, or when viewing AR content such as a navigation type game, the AR unit 12 Replace and install. Since the VR unit 11 and the AR unit 12 are configured to be lightweight, they can be carried as a kit even when away from home.

  In the configuration example of FIG. 1, the AR image capture camera 107 a is provided in the AR unit 12, and the VR image capture camera 107 b is provided in the VR unit 11. This is because the image recognition function and the number of cameras are different between AR and VR. However, there is no problem even if the image capturing camera is provided not in the VR unit 11 and the AR unit 12 but in the AR / VR common unit 10.

  The AR unit 12 is provided with a depth camera unit 108. The depth camera unit 108 is formed by a CCD sensor, a CMOS sensor, or the like that can capture depth information. By providing the depth camera unit 108, a virtual image can be three-dimensionally arranged in a real environment, and a user's gesture in a three-dimensional space can also be recognized, so that it also has a controller function. it can.

  FIG. 2 shows a configuration example for detaching the VR unit 11 and the AR unit 12 from the AR / VR common unit 10. In FIG. 2, the manner in which the AR unit 12 is attached is illustrated, but the VR unit 11 can be similarly configured.

  The unit engaging portion 14a on the AR unit 12 side includes a right engaging portion 14aR having an engaging hinge portion 18 and a left engaging portion 14aL having a connector portion 16a. The unit engagement portion 14b on the AR / VR common unit 10 side includes a right engagement portion 14bR that engages with the right engagement portion 14aR of the AR unit 12, and a left engagement that engages with the left engagement portion 14aL of the AR unit 12. And a joint portion 14bL. The right engagement portion 14bR has a guide space 19 that receives the engagement hinge portion 18. The left engaging portion 14bL includes a connector portion 16b connected to the connector portion 16a and a magnet attracting portion 15 that attracts the left engaging portion 14aL with a magnetic force. With this configuration, when the engagement hinge portion 18 is inserted into the guide space 19 and the left engagement portion 14aL is engaged with the left engagement portion 14bL, the engagement state is fixed by the magnet attracting portion 15. Yes. In this example, the connector part and the magnet attracting part are provided only in the left engaging part, but the connector part and the magnet attracting part may be provided in the right engaging part. The method of engaging the VR unit 11 and the AR unit 12 with the AR / VR common unit 10 is not limited to the example shown in FIG. In addition, a method of fixing using the elastic force of plastic, a method of fastening both with screws or the like can be considered.

  FIG. 3 shows a side view of the external configuration of the head mounted display when reproducing VR content. FIG. 3 shows only the left side, but the right side also has the same configuration. The head mounted display is used by a user wearing on the head. Since the VR unit 11 is configured so that light from the outside of the display does not enter both the left and right eyes, the display unit having a wide viewing angle improves the user's immersive feeling. The user cannot directly view the actual scenery, but can indirectly view the external video captured by the camera.

  The AR / VR common unit 10 includes a circuit board 5 on which electronic circuits such as signal processing are mounted, an HMD attitude detection unit 102, and a video battery 106. The AR / VR common unit 10 can be stabilized by being supported in the vicinity of both ears by a frame structure such as glasses in order to be fixed to the user's head. The circuit board 5 includes a video signal processing unit, an external communication unit, an AR / VR switching unit, a memory, and the like, which will be described later with reference to FIG. VR video content is received via a wireless LAN (IEEE 802.11a / b / g / n / ac), Bluetooth (registered trademark) 4.0, etc. from the Internet, etc., and is sent to a semiconductor memory or a microSD card. It is also possible to save. The HMD posture detection unit 102 constantly monitors the position state around the user's head, for example, using a 6-axis detection sensor (3-axis gyro, 3-axis acceleration), 3-axis geomagnetic sensor, or the like.

  The VR unit 11 includes a VR display unit 141, a lens unit 142, and an image capturing camera unit 107. The VR display unit 141 is composed of an organic EL or liquid crystal display, and it is preferable to improve the resolution of the video to 4K or more in order to bring it closer to the real world. The VR display unit 141 has a structure in which the display device is separated into left and right, such as a lens of glasses, for example, a case where a 3.5-inch organic EL is composed of two surfaces, or a 5.5-inch like a smartphone. A left-right integrated structure using organic EL is also possible.

  The earphone unit 13 includes a speaker unit 120, a microphone A 121, a microphone B 122, an audio battery 123, and a controller touch panel unit 124. Although only one earphone unit 13 is shown in FIG. 3, both the left and right earphone units 13 may have the same function. The speaker unit 120 can be inserted into the auricle and is connected to the AR / VR common unit 10 wirelessly. The microphone A121 is for noise cancellation, and has the effect of enhancing the sense of immersion by effectively insulating external sound by collecting external noise and electrically generating a phase difference signal. Therefore, an audio battery 123 that operates a circuit having a wireless transmission function and a noise cancellation function is built in the earphone unit 13. The microphone B122 collects a user's voice instruction command, for example, video playback start, stop, and fast-forward for the voice recognition function. Thereby, operation is possible without using a remote control. When the VR content 11 is played back, it is possible to detect external noise and activate the noise canceling function.

  The controller touch panel unit 124 performs a graphic layer cursor or command that appears on the VR playback screen of the VR display unit 141 as a GUI (graphic user interface) by touching the touch panel with a finger. Easy to operate. When operating with a remote control in both hands as seen on a VR game machine, etc., there is no problem as long as it is viewed for a short time, but the remote control is heavy and tired, there is a risk of falling, there is a risk of colliding with furniture For example, the user is not always satisfied. Since the earphone unit 13 can be controlled in the vicinity of the auricle by the controller touch panel unit 124, the user can control the screen with a minimum movement. Moreover, since the earphone unit 13 is lightweight, it is suitable for long-time viewing. If the touch panel 124 for the controller has a hemispherical shape instead of a flat touch surface, for example, it is easy to visually recognize the execution of a GUI cursor or button that is displayed so as to overlap the 360-degree stereoscopic space image. Further, in FIG. 3 and the like, the earphone unit 13 is illustrated as a specific example of the sound reproducing device, but the sound reproducing device may be implemented as a headphone type that covers the entire ear. If the headphone type is used, the controller touch panel can be formed relatively large, so that it is relatively easy to have the same function as described above.

  FIG. 4 shows a side view of the external configuration of the HMD when the AR unit 12 is attached. In the VR unit 11, the video display units arranged on both the left and right eyes are transmissive, so that it is possible to provide a user with a composite experience that makes a virtual object video appear in an actual landscape. The left side is displayed as in FIG. 3, but the right side also has the same configuration.

  The AR unit 12 includes an AR display unit 132, a reflecting surface 133, a light guide plate 134, an opaque filter 135, a lens unit 136, and an image capturing camera unit 107.

  The AR display unit 132 emits a virtual image video source. The image is reflected by the reflecting surface 133 and reaches the user's eyes via the light guide plate 134. As the AR display unit 132, a high-power LED, which is also called a micro display, an organic EL, a DLP (Digital Light Processing) display, or an LCOS (Liquid Crystal On Silicon) in which a liquid crystal and a drive circuit are integrated is used. Although the light guide plate 134 is substantially transparent, a diffraction grating such as a holographic film is provided on the surface thereof, and functions as a screen of a video projector by expanding light from the AR display unit 132.

  The opaque filter 135 removes natural light in order to improve the contrast of the image of the virtual object video. The opaque filter 135 is aligned with the light guide plate 134 to selectively transmit or block natural light from the front of the HMD for the entire screen or for each pixel to guide the light to the eye. For example, when rendering a virtual image, the image is displayed as if the virtual image actually exists by adjusting the transparency of the opaque filter 135 according to the positional relationship between the virtual image and the real image. Can do. The lens unit 136 is a standard transparent type lens that is generally used for eyeglasses. The image capturing camera unit 107 has an environment recognition function.

  The microphone B122 can collect a command instructed by the user as a voice input function using a microphone. Further, the microphone B122 can also be used to actively take in the sound of the external environment so that the external sound can be heard even when the earphone unit 13 is inserted into the auricle. Note that when the AR content reproduction is started, the microphone B122 may be automatically activated by detecting the metadata of the AR content.

  FIG. 5 is a block diagram showing functional components of the video / audio reproduction device according to the present embodiment. The AR / VR common unit 10 includes a memory unit 100, a reproduction signal processing unit 101, an HMD attitude detection sensor unit 102, a communication unit 104, an AR / VR switching unit 105, a video battery 106, and an image capture camera unit 107. ing.

  The VR unit 11 includes a VR reproduction I / F (interface) unit 113, a VR display driving unit 140, and a VR display unit 141. When playing back VR content, as shown in FIG. 1, the VR unit 11 is attached to the AR / VR common unit 10 so that the user can experience the immersive video world.

  The AR / VR switching unit 105 determines whether the content to be reproduced is AR / VR. Here, as a technique for determining whether the content to be reproduced is AR or VR, for example, (1) detecting whether the AR unit 12 or the VR unit 11 is attached to the AR / VR common unit 10 And (2) a method of detecting which reproduction of AR / VR content is selected by the user on the initial screen or the like. That is, in the case of the above (1), the VR unit 11 and the AR unit 12 are provided with a mechanism for generating a signal capable of identifying the unit, or the shape of the unit engaging portion 14 in the VR unit 11 and the AR unit 12 is mutually different. It may be possible to detect which of the VR unit 11 and the AR unit 12 is mounted on the AR / VR common unit 10 side by making the difference.

  When it is determined whether the content to be reproduced is AR or VR, the VR reproduction I / F unit 113 displays a thumbnail of the reproducible VR content on the VR display unit 141, thereby allowing the VR content to be displayed by the user. Can be selected. When the VR unit 11 is used, only the VR content can be selected from the content stored in the memory unit 100 and can be displayed and reproduced.

  The AR unit 12 includes an AR reproduction I / F unit 112, an AR display driving unit 130, an opaque filter control unit 131, and an AR display unit 132. At the time of reproduction of AR content, as shown in FIG. 4, by attaching the AR unit 12 to the AR / VR common unit 10, a plurality of virtual images can be seamlessly fused to the viewpoint of the real world user, A mixed reality experience can be provided to the user.

  An instruction relating to a video recorded in the memory unit 100 or a virtual image obtained from the communication unit 104 is received, and a user's head is obtained by a 3-axis gyro, 3-axis acceleration, 3-axis geomagnetic sensor obtained from the HMD posture detection sensor unit 102. Position information centered on the unit is passed to the signal processing unit 101. The signal processing unit 101 determines when and where the virtual image is provided.

  The image received from the image capture camera 107 is stored in the memory unit 100. The AR display driving unit 130 drives the AR display unit 132 (micro display). Information on the virtual image displayed on the AR display unit 132 is provided to the opacity filter control unit 131 that controls the opacity filter. At this time, the video is buffered in the AR reproduction I / F unit 112 to provide system timing data, and a timing circuit and a buffer memory circuit are configured.

  When the AR / VR switching unit 105 identifies that the AR unit 12 is attached to the AR / VR common unit 10 or when the user selects reproduction of AR content, the AR reproduction I / F unit 112 performs reproduction. By displaying thumbnails of possible AR contents on the AR display unit 132, the user can select the AR contents to be reproduced. That is, when the AR unit 12 is used, only the AR content stored in the memory unit 100 can be displayed and reproduced.

  The earphone unit 13 includes an audio amplifier / DAC unit 110, an audio processing unit 111, an earphone speaker unit 120, a microphone A 121, a microphone B 122, an audio battery 123, and a controller touch panel unit 124. The earphone unit 13 can wirelessly communicate with the communication unit 104, but can also be connected via a wire cable.

  The Bluetooth codec such as AAC or aptX transmitted from the communication unit 104 is decoded by the audio amplifier / DAC unit 110 and the sound is output from the earphone speaker unit 120. When the noise processing function is activated, the sound processing unit 111 collects external noise from the microphone A 121 and electrically generates a phase difference signal. In addition, the microphone B122 collects a voice instruction command instructed by the user as a voice recognition function.

  When the AR / VR switching unit 105 detects that the VR unit 11 is attached to the AR / VR common unit 10 or when the user selects playback of VR content, the microphone A 121 automatically detects external noise. And the noise canceling function can be activated. When the AR unit 12 is detected, the microphone B122 can be automatically activated.

  Further, the control panel can be easily operated by performing a touch operation with a finger on the touch panel in order to execute a graphic layer cursor or command that appears on the VR playback screen as a GUI (graphic user interface).

[Second Embodiment]
FIG. 6 is a side view of the HMD device when the AR unit according to the second embodiment is mounted. FIG. 7 is a block diagram illustrating components of the HMD of the second embodiment.

  The AR / VR common unit 10 includes an HMD attitude detection sensor 102, an image capture camera unit 107, a video battery 106, an AR / VR switching unit 105, and a memory unit 100a. The earphone unit 13 includes a circuit board 5 as shown in FIG. As shown in FIG. 7, a communication unit 104, a memory unit 100 b, and a reproduction signal processing unit 101 are mounted on the circuit board 5. The communication unit 104, the memory unit 100, and the reproduction signal processing unit 101 are circuits mounted on the AR / VR common unit 10 in the first embodiment, as shown in FIG.

  Since the AR / VR unit 10 and the earphone unit 13 are wirelessly connected, the memory unit 100a of the AR / VR common unit 10 is used as a buffer memory for video data for communication. The memory unit 100b of the earphone unit 13 is an external memory such as a microSD card that stores video content.

  When reproducing the AR content, the communication unit 104 of the earphone unit 13 downloads and stores the AR video content received via the Internet or the like in the memory unit 100b. The downloaded content is decoded by the reproduction signal processing unit 101. The decoded video signal is transmitted and stored in the memory unit 100 a of the AR / VR common unit 10 via the communication unit 104. The object video is displayed on the AR display unit 132 via the AR playback I / F unit 112 of the AR unit 12 in a state where a sufficient amount of data is secured to play back the video. The audio signal is reproduced in synchronization with the video signal by the earphone speaker unit 120 via the audio amplifier and the DAC unit 110.

  Similarly, during playback of VR content, information determined by the AR / VR switching unit 105 of the earphone unit 13 is transmitted from the playback signal processing unit 101 to the communication unit 104 and displayed on the VR display unit 141 of the VR unit 11. . Signals from the HMD posture detection sensor unit 102 and the image capture camera unit 107 are calculated and processed by the reproduction signal processing unit 101 of the earphone unit 13 via the communication unit 104.

  That is, in this embodiment, the AR / VR common unit 10 can be reduced in weight by incorporating the circuit board 5 in which the signal processing circuit, the memory circuit, and the like are integrated in the earphone unit 13. Thereby, it does not get tired even for long-time viewing.

  Further, when the earphone unit 13 is lost, the communication unit 104 obtains the position information of the paired earphone unit 13 and displays the position of the earphone unit in a three-dimensional manner on the display of the VR unit 11 or the AR unit 12. It is possible to investigate.

  FIG. 8 is a conceptual diagram when the position of the earphone unit 13 is detected. The earphone unit 13 includes a beacon terminal 150 (not shown in FIG. 7) in addition to the configuration shown in FIG. Further, the AR / VR common unit 10 includes two detection units 151 and 152 (not shown in FIG. 7) at positions corresponding to the left and right sides of the user's face. If the earphone unit 13 is lost, if the power of the earphone unit 13 is turned on, the position information transmitted by the communication unit 104 is identified by a smartphone or PC via the Internet using, for example, the GPS function You can also However, in many cases, the earphone is lost in a place where the GPS function does not work, such as indoors, or the power of the earphone main body is often turned off. The beacon terminal 150 can be used for about one year even with a button battery, and can be detected at a short distance by being incorporated in the earphone unit 13. In the present embodiment, two detection units 151 and 152 are arranged on the left and right of the AR / VR common unit 10, and therefore by calculating the difference between signals detected by the detection units 151 and 152, The position can be specified more accurately. In particular, since the detectors 151 and 152 are arranged on the left and right sides of the user's head, the detectors 151 and 152 are affected by radio wave interference by the user's head, so that one of the detectors 151 and 152 can obtain stronger signal strength. .

  FIG. 9 shows the relationship between the distance from the beacon terminal 150 built in the earphone unit 13 to the detection units 151 and 152 and the radio wave intensity RSSI (Received Signal Strength Indication). For example, as shown in FIG. 9, when the detection value by the right detection unit 152 is larger than the detection value by the left detection unit 151, it can be seen that the earphone unit 13 is on the right side of the user's head. Further, from this state, if the detection value by the detection unit 151 becomes equal to the detection value by the detection unit 152 when the user turns his head to the right side, the beacon terminal 150 is determined to be in front of the user. be able to. In this state, the distance to the beacon terminal 150 can be calculated based on the strength of the beacon signal.

  FIG. 10 shows an example in which the distance and direction from the earphone unit 13 incorporating the beacon terminal 150 are displayed in text on the display. Since the direction and the distance are displayed, the user can find the earphone unit 13 in a short time.

  FIG. 11 shows an example in which the distance and direction from the earphone unit 13 including the beacon terminal 150 are graphically displayed on the display. The direction of the earphone unit 13 is indicated by an arrow, and the user can find the earphone unit 13 sensuously by displaying the position of the earphone unit 13 like a ripple when approaching.

  Note that the display modes illustrated in FIGS. 10 and 11 are merely examples, and any display mode can be employed.

  The earphone unit 13 illustrated so far is a cordless complete wireless form, but it is also conceivable that the circuit board 5 and the audio battery 123 are separated from the earphone unit 13 and connected by a cable. It is also conceivable that the HMD posture detection sensor 102 and the like are built in the earphone unit 13 by further integration. The video battery 106 can also be integrated with the audio battery 123 of the earphone unit 13 if wireless power feeding technology becomes possible, further reducing the weight of the HMD device and providing a comfortable viewing environment. realizable.

[Third embodiment]
FIG. 12 shows the HMD device according to the third embodiment, and is a side view showing a state where the AR unit 12 is mounted. FIG. 13 is a block diagram showing a functional configuration of the third embodiment.

  The earphone unit 13 of this embodiment is mechanically and electrically coupled to the VR unit 12 and the AR unit 11. Therefore, the video and audio batteries can be integrated and stored in the earphone unit 13 as the video and audio battery 125. That is, in this embodiment, the video battery 106 and the audio battery 123 are omitted, and the video and audio batteries 125 are mounted on the earphone unit 13, thereby further reducing the weight of the HMD device. Similarly to the second embodiment, the circuit board 5 on which the signal processing circuit, the memory circuit, and the like are integrated, and the HMD posture detection sensor unit 102 are also built in the earphone unit 13. As a result, the weight of the HMD device can be reduced, so that even a long-time viewing does not get tired. Moreover, since it is not wireless, there is little worry that the earphone unit 13 is lost.

  The left and right earphone units 13 include an HMD attitude detection sensor 102, a battery 125, a main circuit board 5, a microphone A121, a microphone B122, a speaker unit 120, a controller touch panel unit 124, and the like. Even if the component configurations of the left and right earphone units 13 are asymmetric, there is no problem as long as the right and left weight distribution is constructed.

  Further, since the earphone unit 13 is not a wireless earphone, it is not necessary to decode the audio signal by the communication unit 104 unlike the above-described embodiment in which audio is received wirelessly such as Bluetooth.

  The technology disclosed in this specification has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the scope of the technology disclosed in this specification.

  For example, the input device is not limited to the controller touch panel unit 124 of the earphone unit 13. A microphone or an infrared sensor may be used as the input device.

10 AR / VR common unit 11 VR unit 12 AR unit 13 Earphone unit 14 Unit engaging part 107 Image chapter camera part 108 Depth camera part 136 Lens part 141 VR display part

Claims (14)

A playback circuit unit capable of playing back virtual reality content and augmented reality content;
A control circuit unit that selects and reproduces either the virtual reality content or the augmented reality content on the reproduction circuit unit;
A first display unit displaying an image of virtual reality content;
A second display unit displaying an image of augmented reality content;
A video and audio reproduction apparatus comprising a head mount mechanism capable of attaching and detaching and replacing the first display unit and the second display unit. A detection means for detecting which of the first display unit and the second display unit is attached to the head mount mechanism;
When it is detected that the first display unit is mounted, the control circuit unit displays a selection screen of virtual reality content that can be played back by the playback circuit unit on the first display unit,
The control circuit unit, when detecting that the second display unit is mounted, causes the second display unit to display an augmented reality content selection screen that can be played back by the playback circuit unit. The video / audio reproduction device according to 1. Further comprising a sound reproducing device that can be deployed in the vicinity of at least one of the left and right auricles or the ear canal,
The video / audio reproduction device according to claim 1, wherein the control circuit unit is provided in the audio reproduction device.   The video / audio reproduction device according to claim 3, wherein the audio reproduction device includes an input device for inputting an operation instruction to the control circuit unit. The sound reproduction device further includes a position transmission unit that transmits a position signal indicating a current position of the sound reproduction device,
The video and audio reproduction device further includes a position detection unit that detects a position signal transmitted from the position transmission unit,
The control circuit unit causes the first display unit or the second display unit to display the position of the sound reproduction device based on the position signal detected by the position detection unit. The audiovisual reproduction apparatus as described.   The video and audio reproduction device according to claim 5, wherein the position transmission unit transmits a beacon signal as the position signal.   The video / audio reproduction device according to claim 6, wherein the position detection units are provided on both left and right sides of the video / audio reproduction device.   The distance to the sound reproduction device detected by the position detection unit provided on the left side of the video / audio reproduction device and the sound reproduction device detected by the position detection unit provided on the right side of the video / audio reproduction device The video and audio reproduction device according to claim 7, wherein the control circuit unit detects the position of the audio reproduction device using a difference from the distance.   The said control circuit part displays the distance to the said position transmission part which is a transmission source of the said position signal with a text on said 1st display part or a 2nd display part, Any one of Claims 6-8. The audiovisual reproduction apparatus according to the item.   The said control circuit part displays the direction of the said position transmission part which is a transmission source of the said position signal with a graphic on said 1st display part or a 2nd display part, Any one of Claims 6-9. The video and audio reproduction device described in 1.   The video and audio reproduction device according to any one of claims 1 to 10, further comprising a first microphone capable of detecting an external noise signal and a second microphone capable of detecting an external audio signal. When the playback circuit unit is playing back virtual reality content, the first microphone operates preferentially,
The video / audio reproduction device according to claim 11, wherein the second microphone operates preferentially when the reproduction circuit unit reproduces augmented reality content. An audio reproduction device that is used together with a video and audio reproduction device including a reproduction circuit unit capable of reproducing virtual reality content and augmented reality content, and that can be deployed in the vicinity of at least one of the left and right auricles or the ear canal,
An acoustic reproduction apparatus comprising a control circuit unit that selects any one of the virtual reality content and the augmented reality content and causes the reproduction circuit unit to reproduce the selected content.   The audio reproduction device according to claim 13, comprising a control circuit and a sensor used for operating the video and audio reproduction device.