CN114779470A - Display method of augmented reality head-up display system - Google Patents

Display method of augmented reality head-up display system Download PDF

Info

Publication number
CN114779470A
CN114779470A CN202210259144.3A CN202210259144A CN114779470A CN 114779470 A CN114779470 A CN 114779470A CN 202210259144 A CN202210259144 A CN 202210259144A CN 114779470 A CN114779470 A CN 114779470A
Authority
CN
China
Prior art keywords
height
microcontroller
driver
vehicle
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210259144.3A
Other languages
Chinese (zh)
Inventor
严小天
于洋
王惠青
温茹涵
苏学前
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Virtual Reality Research Institute Co ltd
Original Assignee
Qingdao Virtual Reality Research Institute Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Virtual Reality Research Institute Co ltd filed Critical Qingdao Virtual Reality Research Institute Co ltd
Priority to CN202210259144.3A priority Critical patent/CN114779470A/en
Publication of CN114779470A publication Critical patent/CN114779470A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0149Head-up displays characterised by mechanical features
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0149Head-up displays characterised by mechanical features
    • G02B2027/0154Head-up displays characterised by mechanical features with movable elements

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Instrument Panels (AREA)

Abstract

The invention discloses a display method of an augmented reality head-up display system, which belongs to the technical field of head-up display systems and comprises a microcontroller, a display module, a reflector, an internal camera, an external camera and a height adjusting mechanism, wherein the microcontroller is used for controlling the height of the display module; the display method comprises the following steps: s1, igniting the vehicle, and initializing the system; s2, the microcontroller acquires road information in front of the vehicle through an external camera and displays corresponding auxiliary information through the display module; s3, the microcontroller collects OBD data of the vehicle, analyzes the OBD data into corresponding vehicle information, and displays the vehicle information through the display module; s4, the microcontroller acquires the eye height of the driver through an internal camera; s5, the microcontroller adjusts the projection height of the reflector through the height adjusting mechanism according to the eye height of the driver, so that the situation that the background with the excessively high enhanced surface is sky or the background with the excessively low state surface is the bonnet when the driver watches the vehicle due to different heights is avoided, the projection surface is obvious and easy to watch, and the driving experience is improved.

Description

Display method of augmented reality head-up display system
Technical Field
The invention belongs to the technical field of head-up display systems, and particularly relates to a display method of an augmented reality head-up display system.
Background
A HUD (Head Up Display) originally comes to the fighter, which can reduce the frequency of the pilot looking down at the instrument panel and avoid the pilot from inattentive driving. Subsequently, since the HUD device brings some convenience while the cost is further reduced, the automobile also starts to follow up slowly. Through the HUD device, the user can observe information such as vehicle speed, speed limit indication, driving route map, and the like. In contrast to the HUD, AR-HUD not only can display driving information, but also changes the way the driver views the information.
The AR-HUD (augmented reality head-up display) can reasonably display some driving information in a superimposed manner in a sight area of a driver and is combined with actual traffic road conditions. By AR-HUD technology, the driver can extend and enhance his own perception of the driving environment. In an AR-HUD, two projection planes based on different projection distances may be generated, referred to as the "proximal" (or "state") plane and the "distal" (or "enhancement") plane, respectively. The projection of state plane of projection shows at driver's the place ahead distance of about 3 meters, and the projection of reinforcing plane of projection shows at driver's the place ahead distance of about 10 meters, and prior art center throws the height and is fixed, and when driver's height was too high or low, can lead to the plane of projection near too much or too far away when seeing the plane of projection, lead to the impression relatively poor, and the background of plane of projection can influence the observation effect, and the mistake is seen easily, and driving experience is poor.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the display method of the augmented reality head-up display system is provided, the projection height of the projection plane can be adjusted according to the positions of the eyes of a driver, and the projection plane is obvious and easy to watch, so that the driving experience is improved.
In order to solve the technical problem, the technical scheme of the invention is as follows:
a display method of an augmented reality head-up display system comprises a microcontroller, a display module, a reflector, an internal camera, an external camera and a height adjusting mechanism; the display method comprises the following steps: s1, igniting the vehicle, and initializing the system; s2, the microcontroller acquires the road information in front of the vehicle through the external camera and displays the corresponding auxiliary information through the display module; s3, the microcontroller collects OBD data of the vehicle, analyzes the OBD data into corresponding vehicle information, and displays the vehicle information through the display module; s4, the microcontroller acquires the eye height of the driver through the internal camera; and S5, adjusting the projection height of the reflector by the microcontroller through the height adjusting mechanism according to the height of the eyes of the driver.
Further, the S5 specifically includes the following steps: s5a, judging whether the eye height of the driver is matched with the current height of the projection surface, if so, the height adjusting mechanism does not act; s5b, if not, judging whether the eyes of the driver are higher than the position matched with the current height of the projection surface; s5c, when the height of the eyes of the driver is higher than the position matched with the current height of a projection surface, the projection surface is upwards adjusted; and S5d, when the height of the eyes of the driver is lower than the position matched with the current height of the projection surface, adjusting the projection surface downwards.
Further, in S3, the internal camera captures an image of the head of the driver, and the microcontroller performs eye positioning by analyzing the image.
Further, the microcontroller determines the height of the eye from the pixel distribution of the image.
Further, the microcontroller decodes the image into an information stream of the three primary color light mode, and then analyzes the height of the eyes on the image.
Further, in S3, the internal camera is an infrared camera, and the microcontroller collects the head position of the driver through the infrared camera, and then calculates the height of the eyes according to a height scale model of the eyes on the head.
Furthermore, the auxiliary information includes navigation, road condition prompt and lane line.
Furthermore, in S2 the OBD data acquisition process is to read vehicle computer data through on-vehicle OBD interface.
Further, the vehicle information comprises the real-time vehicle speed, the engine rotating speed, the water temperature, the oil consumption and the driving mileage of the vehicle.
After the technical scheme is adopted, the invention has the beneficial effects that:
according to the display method of the augmented reality head-up display system, the head-up display system comprises a microcontroller, a display module, a reflector, an internal camera, an external camera and a height adjusting mechanism; the display method comprises the following steps: s1, igniting the vehicle, and initializing the system; s2, the microcontroller acquires the road information in front of the vehicle through the external camera and displays the corresponding auxiliary information through the display module; s3, the microcontroller collects OBD data of the vehicle, analyzes the OBD data into corresponding vehicle information, and displays the vehicle information through the display module; s4, the microcontroller acquires the eye height of the driver through an internal camera; s5, the microcontroller adjusts the projection height of the reflector through the height adjusting mechanism according to the eye height of the driver, and adjusts the height of the projection surface according to the eye height of the driver, so that the situation that the enhanced surface is too high when the driver watches due to different heights, if the background is sky, or the state surface is too low, if the background is a front cover, the projection surface is obvious and easy to watch is avoided, and the driving experience is improved.
Drawings
Fig. 1 is a schematic flow chart of a display method of an augmented reality head-up display system according to the present invention.
Detailed Description
The invention is further illustrated by the following examples in conjunction with the drawings.
All directions referred to in the present specification are based on the drawings, and represent relative positional relationships only, and do not represent absolute positional relationships.
As shown in fig. 1, a display method of an augmented reality head-up display system includes a microcontroller, a projector, an internal camera, an external camera, and a height adjustment mechanism, where the height adjustment mechanism may be a combination structure of a micro motor and a gear reducer. The display method comprises the following steps:
and S1, igniting the vehicle and initializing the system.
When the vehicle is ignited, namely the vehicle is started through a key or a button, the head-up display system is automatically started without manual operation of a driver.
S2, the microcontroller acquires the road information ahead of the vehicle through the external camera, and displays corresponding auxiliary information through the display module, wherein the auxiliary information comprises navigation, road condition prompt, lane lines and the like, and can assist the driving of the driver, and the auxiliary information can be displayed on an enhanced projection surface and is projected and displayed at a distance of about 10 meters ahead of the driver.
And S3, the microcontroller collects OBD data of the vehicle, analyzes the OBD data into corresponding vehicle information, and displays the vehicle information through the display module.
In the embodiment, the OBD data acquisition process is to read the data of the vehicle computer through the vehicle-mounted OBD interface, and the vehicle information comprises the real-time speed of the vehicle, the rotating speed of an engine, the water temperature, the oil consumption and the mileage. The vehicle information can be displayed on a state projection surface, is positioned at the lower end of a front windshield, is projected and displayed in the front of a driver by about 3 meters, does not need to be lowered, and is convenient to watch various information of the vehicle.
And S4, the microcontroller acquires the eye height of the driver through the internal camera.
As shown in fig. 1, one embodiment of the method for obtaining the eye height of the driver in S4 is: the inside camera shoots the head image of the driver, and the microcontroller carries out eye positioning through image analysis. The method is specifically divided into two types, wherein one embodiment is that the microcontroller determines the height of the eye according to the pixel distribution of the image, the other embodiment is that the microcontroller decodes the image into an information stream of a three primary color light mode, and then analyzes the height of the eye on the image through a proper computer algorithm, and the embodiment preferably uses open source OpenCV to detect the height of the eye on the picture. The full name of OpenCV is: open Source Computer Vision Library. OpenCV is a (open source) based cross-platform computer vision library that can run on Linux, Windows, and Mac OS operating systems. OpenCV is composed of a series of C functions and a small number of C + + classes, provides interfaces of languages such as Python, Ruby, MATLAB and the like, and realizes a plurality of general algorithms in the aspects of image processing and computer vision.
As shown in fig. 1, another embodiment of the method for obtaining the eye height of the driver in S4 is as follows: the inside camera is infrared camera, and microcontroller gathers driver's head position through infrared camera, then calculates eye height according to the high proportion model of people's eye at the head, and the proportion model accessible different heights driver measures and reachs.
S5, the microcontroller adjusts the projection height of the reflector through the height adjusting mechanism according to the eye height of the driver, so that the enhanced surface and the state surface are projected and displayed at a proper position in front of the driver, namely the state projection surface is projected and displayed at a distance of about 3 meters in front of the driver, the enhanced projection surface is projected and displayed at a distance of about 10 meters in front of the driver, and the projection surface is obvious and easy to watch, thereby improving the driving experience. The matching model of the height of the eyes of the driver and the height of the projection surface can be obtained through a plurality of tests.
As shown in fig. 1, S5 specifically includes the following steps:
s5a, judging whether the eye height of the driver is matched with the current height of the projection surface, if so, stopping the height adjusting mechanism;
s5b, if not, judging whether the eye height of the driver is higher than the position matched with the current height of the projection surface;
s5c, when the height of the eyes of the driver is higher than the position matched with the current height of the projection surface, the projection surface is adjusted upwards;
and S5d, when the height of the eyes of the driver is lower than the position matched with the current height of the projection surface, the projection surface is adjusted downwards.
The display method of the augmented reality head-up display system can adjust the projection height of the projection surface according to the height of the eyes of the driver, so that the augmented surface and the state surface are projected and displayed at proper positions in front of the driver, the projection surface is obvious and easy to watch, the eyes are not tired, and the driving experience can be improved.
While specific embodiments of the invention have been described, it will be understood by those skilled in the art that the described embodiments are merely illustrative of some, and not restrictive, of the invention, which is defined by the following claims. Various changes or modifications to these embodiments may be made by those skilled in the art without departing from the principle and spirit of the invention, and these changes and modifications all fall within the scope of the invention.

Claims (9)

1. A display method of an augmented reality head-up display system is characterized in that the head-up display system comprises a microcontroller, a display module, a reflector, an internal camera, an external camera and a height adjusting mechanism;
the display method comprises the following steps:
s1, igniting the vehicle, and initializing the system;
s2, the microcontroller acquires road information in front of the vehicle through the external camera and displays corresponding auxiliary information through the display module;
s3, the microcontroller collects OBD data of the vehicle, analyzes the OBD data into corresponding vehicle information, and displays the vehicle information through the display module;
s4, the microcontroller acquires the eye height of the driver through the internal camera;
and S5, adjusting the projection height of the reflector by the height adjusting mechanism according to the eye height of the driver through the microcontroller.
2. The display method of the augmented reality head-up display system according to claim 1, wherein the S5 specifically includes the following steps:
s5a, judging whether the eye height of the driver is matched with the current height of the projection surface, if so, the height adjusting mechanism does not act;
s5b, if not, judging whether the eyes of the driver are higher than the position matched with the current height of the projection surface;
s5c, when the height of the eyes of the driver is higher than the position matched with the current height of a projection surface, upward adjustment of the projection surface is carried out;
and S5d, when the height of the eyes of the driver is lower than the position matched with the current height of the projection surface, adjusting the projection surface downwards.
3. The display method of the augmented reality heads-up display system of claim 1, wherein the internal camera captures an image of the driver' S head in the S3, and the microcontroller performs eye positioning by analyzing the image.
4. The display method of the augmented reality heads-up display system of claim 3 wherein the microcontroller determines the height of the eye from the pixel distribution of the image.
5. The method of claim 3, wherein the microcontroller decodes the image into an information stream of three primary color light modes and then analyzes the eye height on the image.
6. The display method of the augmented reality head-up display system of claim 1, wherein the internal camera in S3 is an infrared camera, and the microcontroller collects the head position of the driver through the infrared camera and then calculates the eye height according to a model of the eye height ratio on the head.
7. The method of claim 1, wherein the auxiliary information comprises navigation, road condition prompts, lane markings.
8. The display method of the augmented reality head-up display system according to any one of claims 1 to 7, wherein the OBD data collection process in S2 is to read vehicle computer data through a vehicle-mounted OBD interface.
9. The display method of the augmented reality heads-up display system of claim 8 wherein the vehicle information includes vehicle real-time speed, engine speed, water temperature, fuel consumption, mileage.
CN202210259144.3A 2022-03-16 2022-03-16 Display method of augmented reality head-up display system Pending CN114779470A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210259144.3A CN114779470A (en) 2022-03-16 2022-03-16 Display method of augmented reality head-up display system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210259144.3A CN114779470A (en) 2022-03-16 2022-03-16 Display method of augmented reality head-up display system

Publications (1)

Publication Number Publication Date
CN114779470A true CN114779470A (en) 2022-07-22

Family

ID=82425281

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210259144.3A Pending CN114779470A (en) 2022-03-16 2022-03-16 Display method of augmented reality head-up display system

Country Status (1)

Country Link
CN (1) CN114779470A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115886816A (en) * 2022-11-15 2023-04-04 立讯精密科技(南京)有限公司 Fatigue detection method based on VR/AR equipment and VR/AR equipment

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005066023A (en) * 2003-08-25 2005-03-17 Nissan Motor Co Ltd Apparatus for detecting driver's condition
JP2006301199A (en) * 2005-04-19 2006-11-02 Konica Minolta Photo Imaging Inc Photographing apparatus
KR20100026466A (en) * 2008-08-29 2010-03-10 엘지전자 주식회사 Head up display system and method for adjusting video display angle thereof
CN101916496A (en) * 2010-08-11 2010-12-15 无锡中星微电子有限公司 System and method for detecting driving posture of driver
CN106226910A (en) * 2016-09-08 2016-12-14 邹文韬 HUD system and image regulating method thereof
CN106483659A (en) * 2015-08-24 2017-03-08 福特全球技术公司 Method for observing the eyes of the driver of the vehicle with HUD
CN106814458A (en) * 2017-02-13 2017-06-09 惠州市华阳多媒体电子有限公司 A kind of augmented reality HUD
CN108352075A (en) * 2015-11-06 2018-07-31 欧库勒斯虚拟现实有限责任公司 It is tracked using the eyes of light stream
CN108437999A (en) * 2018-03-20 2018-08-24 中国计量大学 A kind of attention auxiliary system
CN109353279A (en) * 2018-12-06 2019-02-19 延锋伟世通电子科技(上海)有限公司 A kind of vehicle-mounted head-up-display system of augmented reality
DE102017215161A1 (en) * 2017-08-30 2019-02-28 Volkswagen Aktiengesellschaft Method and device for selecting an environment object in the environment of a vehicle
WO2019088661A1 (en) * 2017-10-30 2019-05-09 코오롱인더스트리 주식회사 Device, system and method for providing service relating to advertising and product purchase by using artificial-intelligence technology
WO2019224922A1 (en) * 2018-05-22 2019-11-28 三菱電機株式会社 Head-up display control device, head-up display system, and head-up display control method
US10510276B1 (en) * 2018-10-11 2019-12-17 Hyundai Motor Company Apparatus and method for controlling a display of a vehicle
WO2020049873A1 (en) * 2018-09-05 2020-03-12 マクセル株式会社 Head-up display device
CN110996051A (en) * 2019-11-26 2020-04-10 斑马网络技术有限公司 Method, device and system for vehicle vision compensation
CN112639580A (en) * 2020-09-14 2021-04-09 华为技术有限公司 Head-up display device, head-up display method and vehicle
CN112912825A (en) * 2018-10-24 2021-06-04 Pcms控股公司 System and method for region of interest estimation for virtual reality
WO2021184932A1 (en) * 2020-03-20 2021-09-23 华为技术有限公司 Method and device for generating anthropomorphic 3d model

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005066023A (en) * 2003-08-25 2005-03-17 Nissan Motor Co Ltd Apparatus for detecting driver's condition
JP2006301199A (en) * 2005-04-19 2006-11-02 Konica Minolta Photo Imaging Inc Photographing apparatus
KR20100026466A (en) * 2008-08-29 2010-03-10 엘지전자 주식회사 Head up display system and method for adjusting video display angle thereof
CN101916496A (en) * 2010-08-11 2010-12-15 无锡中星微电子有限公司 System and method for detecting driving posture of driver
CN106483659A (en) * 2015-08-24 2017-03-08 福特全球技术公司 Method for observing the eyes of the driver of the vehicle with HUD
CN108352075A (en) * 2015-11-06 2018-07-31 欧库勒斯虚拟现实有限责任公司 It is tracked using the eyes of light stream
CN106226910A (en) * 2016-09-08 2016-12-14 邹文韬 HUD system and image regulating method thereof
CN106814458A (en) * 2017-02-13 2017-06-09 惠州市华阳多媒体电子有限公司 A kind of augmented reality HUD
DE102017215161A1 (en) * 2017-08-30 2019-02-28 Volkswagen Aktiengesellschaft Method and device for selecting an environment object in the environment of a vehicle
WO2019088661A1 (en) * 2017-10-30 2019-05-09 코오롱인더스트리 주식회사 Device, system and method for providing service relating to advertising and product purchase by using artificial-intelligence technology
CN108437999A (en) * 2018-03-20 2018-08-24 中国计量大学 A kind of attention auxiliary system
WO2019224922A1 (en) * 2018-05-22 2019-11-28 三菱電機株式会社 Head-up display control device, head-up display system, and head-up display control method
WO2020049873A1 (en) * 2018-09-05 2020-03-12 マクセル株式会社 Head-up display device
US10510276B1 (en) * 2018-10-11 2019-12-17 Hyundai Motor Company Apparatus and method for controlling a display of a vehicle
CN112912825A (en) * 2018-10-24 2021-06-04 Pcms控股公司 System and method for region of interest estimation for virtual reality
CN109353279A (en) * 2018-12-06 2019-02-19 延锋伟世通电子科技(上海)有限公司 A kind of vehicle-mounted head-up-display system of augmented reality
CN110996051A (en) * 2019-11-26 2020-04-10 斑马网络技术有限公司 Method, device and system for vehicle vision compensation
WO2021184932A1 (en) * 2020-03-20 2021-09-23 华为技术有限公司 Method and device for generating anthropomorphic 3d model
CN112639580A (en) * 2020-09-14 2021-04-09 华为技术有限公司 Head-up display device, head-up display method and vehicle

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115886816A (en) * 2022-11-15 2023-04-04 立讯精密科技(南京)有限公司 Fatigue detection method based on VR/AR equipment and VR/AR equipment
CN115886816B (en) * 2022-11-15 2024-05-10 立讯精密科技(南京)有限公司 VR/AR equipment and fatigue detection method based on same

Similar Documents

Publication Publication Date Title
CN107554425B (en) A kind of vehicle-mounted head-up display AR-HUD of augmented reality
US11241960B2 (en) Head up display apparatus and display control method thereof
EP2133728B1 (en) Method and system for operating a display device
US8994558B2 (en) Automotive augmented reality head-up display apparatus and method
CN103010099B (en) Multidate information display on full blast gear head-up display
US7629877B2 (en) Multifunctional collimator indicator
CN111433067A (en) Head-up display device and display control method thereof
US9267808B2 (en) Visual guidance system
US20190361233A1 (en) Vehicle display control device
US20170060235A1 (en) Method of operating a vehicle head-up display
US20200234676A1 (en) On-vehicle display device, on-vehicle display method and vehicle
CN106483659A (en) Method for observing the eyes of the driver of the vehicle with HUD
KR20130089139A (en) Augmented reality head-up display apparatus and method for vehicles
Langner et al. Traffic awareness driver assistance based on stereovision, eye-tracking, and head-up display
CN114779470A (en) Display method of augmented reality head-up display system
CN203267910U (en) Novel navigation equipment on the basis of automobile windshield injection display
CN108896067B (en) Dynamic display method and device for vehicle-mounted AR navigation
CN108152957A (en) A kind of vehicle-mounted head-up-display system and the error calibrating method based on the system
CN2487595Y (en) Projection nevigation equipment for indicating vehicle running direction
US11828947B2 (en) Vehicle and control method thereof
CN207336931U (en) Head-up display device
CN113866989A (en) Head-mounted display device capable of adjusting imaging distance
WO2021186807A1 (en) Information processing device, vehicle, and information processing method
KR100825276B1 (en) Head up display system for vehicle
CN206609442U (en) A kind of HUD and its light show instruction system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination