CN108445624B - Head-mounted display device with augmented reality function - Google Patents
Head-mounted display device with augmented reality function Download PDFInfo
- Publication number
- CN108445624B CN108445624B CN201810125855.5A CN201810125855A CN108445624B CN 108445624 B CN108445624 B CN 108445624B CN 201810125855 A CN201810125855 A CN 201810125855A CN 108445624 B CN108445624 B CN 108445624B
- Authority
- CN
- China
- Prior art keywords
- real
- camera
- image
- screen
- time image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
- Controls And Circuits For Display Device (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a head-mounted display device with an augmented reality function, which comprises a main body, a fixing part for fixing the main body, a processing module, at least one camera and a display screen, wherein the at least one camera is arranged on the outer side of the main body, the display screen is arranged on the inner side of the main body, and the processing module is used for controlling the display screen to display real-time images acquired by VR (virtual reality) files and/or the camera according to a control instruction. The head-mounted display device can realize multiple functions, enables a user to conveniently check the surrounding environment when browsing VR files, facilitates the switching between VR display and AR display, and brings convenience to the user.
Description
Technical Field
The present invention relates to a head-mounted display device having an augmented reality function.
Background
The virtual reality technology is an important direction of the simulation technology, is a collection of various technologies such as the simulation technology, the computer graphics man-machine interface technology, the multimedia technology, the sensing technology, the network technology and the like, and is a challenging advanced subject and research field of cross technologies. Virtual reality technology (VR) mainly includes aspects of simulating environment, perception, natural skills, sensing equipment and the like. The simulated environment is a three-dimensional realistic image generated by a computer and dynamic in real time. Perception means that an ideal VR should have the perception that everyone has. In addition to the visual perception generated by computer graphics technology, there are also perceptions such as auditory sensation, tactile sensation, force sensation, and movement, and even olfactory sensation and taste sensation, which are also called multi-perception. The natural skill refers to the head rotation, eyes, gestures or other human body behavior actions of a human, and data adaptive to the actions of the participants are processed by the computer, respond to the input of the user in real time and are respectively fed back to the five sense organs of the user. The sensing device refers to a three-dimensional interaction device.
Augmented Reality (AR) is a technology for calculating the position and angle of a camera image in real time and adding corresponding images, videos and 3D models, and aims to cover a virtual world on a screen in the real world and perform interaction. This technique was proposed in 1990. With the improvement of the CPU computing capability of portable electronic products, the application of augmented reality is expected to be wider and wider.
The conventional VR device cannot see the outside while enjoying video contents, and the AR device can see the outside but does not have a feeling of being personally on the scene.
Disclosure of Invention
The invention aims to overcome the defects that a head-mounted display device in the prior art is single in function and has many limitations in use, and provides a head-mounted display device with an augmented reality function, which can realize multiple functions, enables a user to conveniently view the surrounding environment when browsing VR files, facilitates the switching between VR display and AR display and brings convenience to the user.
The invention solves the technical problems through the following technical scheme:
the utility model provides a head-mounted display device, its characterized in that, head-mounted display device includes a main part, one is used for fixing the fixed part of main part, a processing module, an at least camera and a display screen, an at least camera is located the outside of main part, the display screen is located the inboard of main part, processing module is used for controlling according to control instruction the display screen shows the real-time image that VR file and/or camera acquireed.
Preferably, the control instructions include one or more of a first instruction, a second instruction, a third instruction, and a fourth instruction,
the processing module is used for controlling the display screen to display the VR file when receiving a first instruction;
the processing module is used for controlling the display screen to display the real-time image acquired by the camera when receiving a second instruction;
the processing module is used for controlling the display screen to display the VR file and displaying a real-time image acquired by the camera in a preset area in the display screen when a third instruction is received;
and the processing module is used for controlling the display screen to display the real-time image acquired by the camera and display the VR file in a preset area in the display screen when receiving the fourth instruction.
Preferably, the display screen includes first screen and second screen, first screen correspondence sets up in the sight place ahead of left eyeball, the second screen correspondence sets up in the sight place ahead of right eyeball, at least one camera is including being close to the left camera of first screen and being close to the right camera of second screen, and two cameras are located respectively the main part both ends of controlling in the outside, first screen is used for showing the real-time image that left camera acquireed, the second screen is used for showing the real-time image that right camera acquireed.
Preferably, the head-mounted display device further comprises an intercepting module,
the intercepting module is used for intercepting the real-time image acquired by the camera according to an adjusting instruction according to a preset rule, and the adjusting instruction is used for adjusting the size of an area for intercepting the real-time image;
the processing module is used for controlling the display screen to display the captured real-time images according to the control instruction.
Preferably, the preset rule is:
for the rectangular real-time image acquired by the left camera, intercepting the image from the right side edge of the rectangular real-time image along the length direction of the rectangular real-time image, wherein the intercepted image is rectangular, the width of the intercepted image is the same as that of the rectangular real-time image, and the length of the intercepted image is determined according to an adjusting instruction;
for the rectangular real-time image acquired by the right camera, intercepting the image from the left side edge of the rectangular real-time image along the length direction of the rectangular real-time image, wherein the intercepted image is rectangular, the width of the intercepted image is the same as that of the rectangular real-time image, and the length of the intercepted image is determined according to an adjusting instruction;
the length of the intercepted image obtained by the left camera is the same as that of the intercepted image obtained by the right camera.
Preferably, the at least one camera comprises a left camera close to the first screen and a right camera close to the second screen, the two cameras are respectively arranged at the left end and the right end of the outer side of the main body,
the processing module is used for generating a real-time image with depth information according to the images acquired by the two cameras.
Preferably, the head-mounted display device further comprises an identification module,
the recognition module is used for recognizing the gesture image in the real-time image acquired by the camera and sending a control instruction corresponding to the gesture image to the processing module according to the corresponding relation between the gesture image and the control instruction.
Preferably, the head-mounted display device further includes an infrared beam transmitter and an infrared receiver, the infrared beam transmitter and the infrared receiver are arranged side by side in the main body and are configured to detect depth information of the fishtail, the depth information corresponds to a control quantity indicating a degree, and the processing module is configured to control the control quantity according to the depth information.
Preferably, the head-mounted display device further comprises an identification module,
the identification module is used for identifying the movable target in the VR file and adding the movable target into the real-time image acquired by the camera.
On the basis of the common knowledge in the field, the above preferred conditions can be combined randomly to obtain the preferred embodiments of the invention.
The positive progress effects of the invention are as follows: the head-mounted display device can inform a rescue unit at the first time after a traffic accident, improve the survival probability of passengers, simultaneously save the driving information of the vehicle, and realize the functions and services provided by a luxury automobile through a mobile phone in low-cost vehicle allocation.
Drawings
Fig. 1 is a schematic structural diagram of image display in the viewing direction when the user wears the head-mounted display device.
Detailed Description
The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.
Example 1
Referring to fig. 1, the embodiment provides a head-mounted display device with an augmented reality function, where the head-mounted display device includes a main body, a fixing portion for fixing the main body, a processing module, two cameras, and a display screen.
The display screen comprises a first screen and a second screen, the first screen is correspondingly arranged in front of the sight of the left eyeball, and the second screen is correspondingly arranged in front of the sight of the right eyeball.
The two cameras are arranged on the outer side of the main body, the display screen is arranged on the inner side of the main body, and the processing module is used for controlling the display screen to display real-time images acquired by VR files and/or cameras according to control instructions.
In this embodiment, the display modes on the display screen include 4 types, which are respectively:
the display screen displays the VR file;
the display screen displays a real-time image acquired by the camera;
the display screen displays the VR file and displays a real-time image acquired by the camera in a preset area in the display screen;
the display screen displays the real-time image acquired by the camera and displays the VR file in a preset area in the display screen.
Wherein, the last two kinds of condition are the real-time image that display screen shows VR file and camera and acquireed.
The display mode is determined by the control instruction, and the control instruction can be input by the modes of entity keys, voice and touch keys.
In this embodiment, the control instruction includes a first instruction, a second instruction, a third instruction, and a fourth instruction.
The processing module is used for controlling the display screen to display the VR file when receiving the first instruction.
And the processing module is used for controlling the display screen to display the real-time image acquired by the camera when receiving the second instruction.
And the processing module is used for controlling the display screen to display the VR file and displaying the real-time image acquired by the camera in a preset area in the display screen when receiving the third instruction.
And the processing module is used for controlling the display screen to display the real-time image acquired by the camera and display the VR file in a preset area in the display screen when receiving the fourth instruction.
In this embodiment, the predetermined region is a display mode of pip.
Specifically, two cameras are including being close to the left camera of first screen and being close to the right camera of second screen, and two cameras are located respectively the main part both ends of controlling in the outside, first screen is used for showing the real-time image that left camera acquireed, the second screen is used for showing the real-time image that right camera acquireed.
Utilize two cameras to form images respectively, can simulate the visual angle of two eyes, convenience of customers judges with the distance between the object.
The head-mounted display device further comprises an intercepting module.
The intercepting module is used for intercepting the real-time image acquired by the camera according to the adjusting instruction according to the preset rule, and the adjusting instruction is used for adjusting the size of the area for intercepting the real-time image.
The processing module is used for controlling the display screen to display the captured real-time images according to the control instruction.
Because everyone's interpupillary distance is different, the image user who acquires observes probably not to be used to, and this application is through intercepting the image that the camera was shot, the image that the emulation user that can be further acquireed through the eyes.
The preset rule can intercept the image shot by the camera according to the average visual angle of human eyes. In particular, referring to fig. 1, fig. 1 is a schematic structural diagram of image display in an observation direction when a user wears the head-mounted display device, and a specific preset rule provided in this embodiment is as follows:
for a rectangular real-time image 11 (indicated by a dotted line) acquired by a left camera, intercepting the image from the right side edge of the rectangular real-time image along the length direction of the rectangular real-time image, wherein the intercepted image 12 (indicated by a solid line) is rectangular, the width of the intercepted image is the same as that of the rectangular real-time image, and the length of the intercepted image is determined according to an adjusting instruction;
for the rectangular real-time image 13 acquired by the right camera, intercepting the image from the left side edge of the rectangular real-time image along the length direction of the rectangular real-time image, wherein the intercepted image 14 is rectangular, the width of the intercepted image is the same as that of the rectangular real-time image, and the length of the intercepted image is determined according to the adjustment instruction;
the length of the intercepted image obtained by the left camera is the same as that of the intercepted image obtained by the right camera.
After the intercepted image is obtained, the intercepted image is zoomed according to the size of the intercepted area, and the zoom ratio is smaller when the intercepted area is larger.
The head-mounted display device further comprises a measuring module,
the measuring module is used for measuring a length value which is far away from a target object through the two cameras, and the length value is displayed when the real-time image acquired by the cameras is displayed through the display screen.
The real-time image with the depth information can be generated according to the images acquired by the two cameras through the measuring module.
The head-mounted display device also comprises a first identification module,
the first recognition module is used for recognizing the gesture image in the real-time image acquired by the camera and sending a control instruction corresponding to the gesture image to the processing module according to the corresponding relation between the gesture image and the control instruction.
The gesture image can be an image collected by a camera background when the display screen does not display the image shot by the camera.
The head-mounted display device further comprises an infrared beam transmitter and an infrared receiver, the infrared beam transmitter and the infrared receiver are arranged in the main body side by side and used for detecting depth information of the fishtail pattern, the depth information corresponds to a control quantity representing degree, and the processing module is used for controlling the control quantity according to the depth information.
The control quantity of the degree of representation comprises volume intensity, light intensity, fast and slow playing in playing, forward and backward playing and other control quantities related to time, and the control quantity of the degree of representation by blinking can be controlled by a user by using the device.
The head-mounted display device also comprises a second identification module,
the second identification module is used for identifying the movable target in the VR file and adding the movable target into the real-time image acquired by the camera.
According to the embodiment, the movable target in the VR file can be acquired and presented in the AR environment, so that a user can experience that the movable target is located around the user.
Example 2
This embodiment is substantially the same as embodiment 1 except that:
the display screen only comprises one screen, the display area of the screen covers the left eye and the right eye, and the preset area in the display screen is in a picture-in-picture display mode.
The processing module is used for generating a real-time image with depth information according to the images acquired by the two cameras.
Images acquired by the two cameras can generate an image with a naked eye 3D effect, and the real-time images can be better provided for use experience through the screen display.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.
Claims (5)
1. A head-mounted display device with an augmented reality function is characterized by comprising a main body, a fixing part for fixing the main body, a processing module, at least one camera and a display screen, wherein the at least one camera is arranged on the outer side of the main body, the display screen is arranged on the inner side of the main body, and the processing module is used for controlling the display screen to display real-time images acquired by VR (virtual reality) files and/or the camera according to a control instruction;
the display screen comprises a first screen and a second screen, the first screen is correspondingly arranged in front of the sight of the left eyeball, the second screen is correspondingly arranged in front of the sight of the right eyeball, the at least one camera comprises a left camera close to the first screen and a right camera close to the second screen, the two cameras are respectively arranged at the left end and the right end of the outer side of the main body, the first screen is used for displaying a real-time image acquired by the left camera, and the second screen is used for displaying a real-time image acquired by the right camera;
the head-mounted display device also comprises an intercepting module,
the intercepting module is used for intercepting the real-time image acquired by the camera according to an adjusting instruction according to a preset rule, and the adjusting instruction is used for adjusting the size of an area for intercepting the real-time image;
the processing module is used for controlling the display screen to display the intercepted real-time image according to the control instruction;
the preset rule is as follows:
for the rectangular real-time image acquired by the left camera, intercepting the image from the right side edge of the rectangular real-time image along the length direction of the rectangular real-time image, wherein the intercepted image is rectangular, the width of the intercepted image is the same as that of the rectangular real-time image, and the length of the intercepted image is determined according to an adjusting instruction;
for the rectangular real-time image acquired by the right camera, intercepting the image from the left side edge of the rectangular real-time image along the length direction of the rectangular real-time image, wherein the intercepted image is rectangular, the width of the intercepted image is the same as that of the rectangular real-time image, and the length of the intercepted image is determined according to an adjusting instruction;
the length of the intercepted image obtained by the left camera is the same as that of the intercepted image obtained by the right camera;
after the intercepted image is obtained, zooming the intercepted image according to the size of the intercepted area, wherein the larger the intercepted area is, the smaller the zooming ratio is;
the head-mounted display device further comprises an infrared beam transmitter and an infrared receiver, the infrared beam transmitter and the infrared receiver are arranged in the main body side by side and used for detecting depth information of the fishtail pattern, the depth information corresponds to a control quantity representing degree, and the processing module is used for executing the control quantity according to the depth information.
2. The head-mounted display device of claim 1, wherein the control instructions comprise one or more of first instructions, second instructions, third instructions, and fourth instructions,
the processing module is used for controlling the display screen to display the VR file when receiving a first instruction;
the processing module is used for controlling the display screen to display the real-time image acquired by the camera when receiving a second instruction;
the processing module is used for controlling the display screen to display the VR file and displaying a real-time image acquired by the camera in a preset area in the display screen when a third instruction is received;
and the processing module is used for controlling the display screen to display the real-time image acquired by the camera and display the VR file in a preset area in the display screen when receiving the fourth instruction.
3. The head-mounted display apparatus according to claim 1, wherein the at least one camera includes a left camera adjacent to the first screen and a right camera adjacent to the second screen, the two cameras being respectively disposed at left and right ends of an outer side of the main body,
the processing module is used for generating a real-time image with depth information according to the images acquired by the two cameras.
4. The head-mounted display device of claim 1, wherein the head-mounted display device further comprises an identification module,
the recognition module is used for recognizing the gesture image in the real-time image acquired by the camera and sending a control instruction corresponding to the gesture image to the processing module according to the corresponding relation between the gesture image and the control instruction.
5. The head-mounted display device of claim 1, wherein the head-mounted display device further comprises an identification module,
the identification module is used for identifying the movable target in the VR file and adding the movable target into the real-time image acquired by the camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810125855.5A CN108445624B (en) | 2018-02-08 | 2018-02-08 | Head-mounted display device with augmented reality function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810125855.5A CN108445624B (en) | 2018-02-08 | 2018-02-08 | Head-mounted display device with augmented reality function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108445624A CN108445624A (en) | 2018-08-24 |
| CN108445624B true CN108445624B (en) | 2021-01-26 |
Family
ID=63191835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810125855.5A Active CN108445624B (en) | 2018-02-08 | 2018-02-08 | Head-mounted display device with augmented reality function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108445624B (en) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105955456B (en) * | 2016-04-15 | 2018-09-04 | 深圳超多维科技有限公司 | The method, apparatus and intelligent wearable device that virtual reality is merged with augmented reality |
| CN109844735A (en) * | 2016-07-21 | 2019-06-04 | 奇跃公司 | Affective state for using user controls the technology that virtual image generates system |
-
2018
- 2018-02-08 CN CN201810125855.5A patent/CN108445624B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108445624A (en) | 2018-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3117290B1 (en) | Interactive information display | |
| US20110084983A1 (en) | Systems and Methods for Interaction With a Virtual Environment | |
| US9779702B2 (en) | Method of controlling head-mounted display system | |
| US11695908B2 (en) | Information processing apparatus and information processing method | |
| CN110267025B (en) | Rendering method and device for virtual 3D display and display method and system thereof | |
| US11710310B2 (en) | Virtual content positioned based on detected object | |
| US11776182B1 (en) | Techniques for enabling drawing in a computer-generated reality environment | |
| US11212501B2 (en) | Portable device and operation method for tracking user's viewpoint and adjusting viewport | |
| CN113785263A (en) | Virtual model for communication between an autonomous vehicle and an external observer | |
| CN106095106A (en) | Virtual reality terminal and display photocentre away from method of adjustment and device | |
| TW201919390A (en) | Display system and method thereof | |
| JP2010259017A (en) | Display device, display method and display program | |
| US11004273B2 (en) | Information processing device and information processing method | |
| US10602116B2 (en) | Information processing apparatus, information processing method, and program for performing display control | |
| CN106125927B (en) | Image processing system and method | |
| CN108445624B (en) | Head-mounted display device with augmented reality function | |
| JP2019047296A (en) | Display device, display method, control program, and electronic mirror system | |
| CN110796116A (en) | Multi-panel display system, vehicle with multi-panel display system and display method | |
| EP2919094A1 (en) | Interactive information display | |
| CN113552947A (en) | Virtual scene display method and device and computer readable storage medium | |
| JP7258620B2 (en) | Image processing system and image processing method | |
| US10834382B2 (en) | Information processing apparatus, information processing method, and program | |
| US20170302904A1 (en) | Input/output device, input/output program, and input/output method | |
| JP2014071546A (en) | Tactile illusion presenting apparatus and tactile illusion presenting program | |
| US20230412792A1 (en) | Rendering format selection based on virtual distance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |