CN112947761B - Virtual image position adjustment method, device and storage medium of AR-HUD system - Google Patents
Virtual image position adjustment method, device and storage medium of AR-HUD system Download PDFInfo
- Publication number
- CN112947761B CN112947761B CN202110332139.6A CN202110332139A CN112947761B CN 112947761 B CN112947761 B CN 112947761B CN 202110332139 A CN202110332139 A CN 202110332139A CN 112947761 B CN112947761 B CN 112947761B
- Authority
- CN
- China
- Prior art keywords
- virtual image
- gesture
- driver
- hud system
- adjustment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0149—Head-up displays characterised by mechanical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0149—Head-up displays characterised by mechanical features
- G02B2027/0154—Head-up displays characterised by mechanical features with movable elements
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Human Computer Interaction (AREA)
- Instrument Panels (AREA)
Abstract
The embodiment of the application discloses a virtual image position adjustment method, device and storage medium of an AR-HUD system, and belongs to the technical field of intelligent automobile driving. The method comprises the following steps: when a virtual image adjustment gesture of a driver is detected, acquiring a behavior state of the driver and a display state of an AR-HUD system; when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture; and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information. According to the virtual image adjustment gesture of the driver, the virtual image position of the AR-HUD system is adjusted, so that the requirements for different virtual image positions caused by different heights and driving habits of the driver are met, and the safety and the comfort of the driver are improved.
Description
Technical Field
The embodiment of the application relates to the technical field of intelligent automobile driving, in particular to a virtual image position adjusting method, device and storage medium of an AR-HUD (Augmented Reality Head Up Display) system.
Background
With the continuous development of society, automobiles have become an indispensable transportation means for people's daily life. However, as the number of automobiles increases, the frequency of traffic accidents increases. In order to improve driving safety, the automobile can be assisted to drive through an AR-HUD system. The AR-HUD system can project some important information required by a driver during driving onto the front windshield of the automobile, display virtual images and real scenes around the automobile are fused, and the light fused with the real scenes is reflected to absorb eyes of the driver, so that the driver can see the virtual images fused with the real scenes, the driver does not need to look at an instrument in a low head, the eyes look at the road surface in front all the time, and the influence of the low head on safe driving is improved. However, since the driver driving the automobile may change, the height of the driver is not uniform and the driving habit is not uniform, and thus the position requirements for the virtual image are also different, the position of the virtual image needs to be adjusted.
Currently, a hardware switch for adjusting the position of a virtual image can be provided in an automobile, and a driver adjusts the position of the virtual image through the hardware switch. However, because the driver sometimes cannot find the hardware switch, the driving experience of the driver is poor, and the viscosity of the user is reduced.
Disclosure of Invention
The embodiment of the application provides a virtual image position adjustment method, device and storage medium of an AR-HUD system, which can solve the problems of low driving comfort and safety caused by inconvenient adjustment of the virtual image position of the AR-HUD system in the related technology. The technical scheme is as follows:
in one aspect, a method for adjusting a virtual image position of an AR-HUD system is provided, the method comprising:
when a virtual image adjustment gesture of a driver is detected, acquiring a behavior state of the driver and a display state of an augmented reality head-up display (AR-HUD) system;
when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture;
and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In some embodiments, before the step of obtaining the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system when the virtual image adjustment gesture of the driver is detected, the method further includes:
acquiring behavior images of a specified number of drivers through cameras installed on the automobile;
identifying gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and determining that the virtual image adjustment gesture of the driver is detected when the gesture moving direction and the gesture position accord with the virtual image adjustment gesture.
In some embodiments, the obtaining, according to the gesture information of the aerial image adjustment gesture, an aerial image adjustment instruction corresponding to the aerial image adjustment gesture includes:
comparing the gesture information of the virtual image adjustment gesture with stored gesture information of multiple reference gestures;
and when a reference gesture with gesture information matched with the gesture information of the virtual image adjustment gesture exists in the plurality of reference gestures, acquiring the virtual image adjustment corresponding to the reference gesture.
In some embodiments, the adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information includes:
acquiring the current position of the virtual image of the AR-HUD system;
determining an adjustment distance according to the gesture movement distance included in the gesture information;
and adjusting a curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjusting instruction and the adjusting distance so as to adjust the virtual image position of the AR-HUD system, wherein the curved mirror is used for projecting the virtual image of the AR-HUD system.
In some embodiments, the obtaining the current location of the virtual image of the AR-HUD system includes:
acquiring the position and projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
In another aspect, there is provided a virtual image position adjustment apparatus of an AR-HUD system, the apparatus comprising:
the first acquisition module is used for acquiring the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system when the virtual image adjustment gesture of the driver is detected;
the second acquisition module is used for acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state;
and the adjusting module is used for adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In some embodiments, the apparatus further comprises:
the third acquisition module is used for acquiring behavior images of a specified number of drivers through cameras installed on the automobile;
the recognition module is used for recognizing gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and the determining module is used for determining that the virtual image adjustment gesture of the driver is detected when the gesture moving direction and the gesture position accord with the virtual image adjustment gesture.
In some embodiments, the second acquisition module comprises:
the comparison sub-module is used for comparing the gesture information of the virtual image adjustment gesture with stored gesture information of various reference gestures;
the first obtaining sub-module is used for obtaining the virtual image adjusting gesture corresponding to the reference gesture when the reference gesture with gesture information matched with the gesture information of the virtual image adjusting gesture exists in the plurality of reference gestures.
In some embodiments, the adjustment module comprises:
the second acquisition sub-module is used for acquiring the current position of the virtual image of the AR-HUD system;
the determining submodule is used for determining an adjustment distance according to the gesture movement distance included in the gesture information;
and the adjusting sub-module is used for adjusting a curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjusting instruction and the adjusting distance so as to adjust the virtual image position of the AR-HUD system, and the curved mirror is used for projecting the virtual image of the AR-HUD system.
In some embodiments, the second acquisition submodule is to:
acquiring the position and projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
In another aspect, a computer readable storage medium having instructions stored thereon that when executed by a processor perform any of the steps of a virtual image position adjustment method of an AR-HUD system described above is provided.
The beneficial effects that technical scheme that this application embodiment provided include at least:
in this application embodiment, can adjust AR-HUD system's virtual image position through driver's virtual image adjustment gesture, need not to adjust through hardware switch, improved the convenience of virtual image adjustment, satisfy simultaneously and cause the demand to different virtual image positions because of driver's difference in height and driving habit, improved driver's security and travelling comfort.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a virtual image position adjustment system of an AR-HUD system according to an embodiment of the present application;
fig. 2 is a flowchart of a method for adjusting a virtual image position of an AR-HUD system according to an embodiment of the present application;
fig. 3 is a flowchart of a method for adjusting a virtual image position of another AR-HUD system according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a virtual image position adjustment device of an AR-HUD system according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a virtual image position adjustment device of another AR-HUD system according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a second obtaining module according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an adjustment module according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Before explaining a virtual image position adjustment method of an AR-HUD system provided in the embodiment of the present application in detail, an application scenario and a system architecture provided in the embodiment of the present application are described.
Therefore, automobile safety is a major direction of future automobile development. The AR-HUD system is an automobile auxiliary visual safe driving system, and some important information needed by a driver during driving is projected onto front windshield glass of an automobile through an optical path of the AR-HUD system, so that the driver does not need to look at an instrument panel at a low head, eyes watch a front view road surface all the time, and the influence of the low head on safe driving is reduced. Because the drivers driving the automobile may change, the heights of the drivers are inconsistent and the driving habits are inconsistent, so that the position requirements on the virtual images are different, and therefore, the positions of the virtual images need to be adjusted.
Based on such application scenarios, the embodiments of the present application provide a virtual image position adjustment method of an AR-HUD system that improves driving safety and comfort.
Next, a virtual image position adjustment system architecture of an AR-HUD system provided in an embodiment of the present application is described.
Fig. 1 is a schematic diagram of a virtual image position adjustment system architecture of an AR-HUD system according to an embodiment of the present application, referring to fig. 1, the system architecture includes a sound host 1, a camera 2, a library file module 3, an optical machine controller 4, a stepper motor 5, and a curved mirror 6. The sound host 1 is respectively connected with the camera 2, the library file module 3 and the optical machine controller 4, the optical machine controller 4 is also connected with the stepping motor 5, and the stepping motor 5 is connected with the curved mirror 6.
As an example, the camera 2 is configured to acquire a virtual adjustment gesture, a behavior state, and the like of the driver, and to transfer the acquired virtual adjustment gesture and behavior state of the driver to the head set 1. The audio host 1 is configured to receive a virtual adjustment gesture and a behavior state sent by the camera 2, obtain a display state of the AR-HUD system from the optical machine controller 4, obtain a virtual image adjustment instruction corresponding to the virtual image adjustment gesture from the library file module according to gesture information of the virtual image adjustment gesture when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, obtain an adjustment angle and an adjustment distance according to the virtual image adjustment instruction, gesture information and the behavior state of the driver, and send the adjustment angle and the adjustment distance to the optical machine controller 4; the optical machine controller 4 sends a virtual image adjusting instruction, an adjusting position angle and an adjusting distance to the stepping motor 5; the stepper motor 5 is used for adjusting the curved mirror 6 according to the virtual adjustment instruction, the adjustment position angle and the adjustment distance so as to adjust the virtual image position of the AR-HUD system.
Those skilled in the art will appreciate that the above system architecture is merely exemplary, and that other modules, components, etc. that may be present or may appear in the future are applicable to and within the scope of the present application and are incorporated herein by reference.
Fig. 2 is a flowchart of a virtual image position adjustment method of an AR-HUD system according to an embodiment of the present application, where the virtual image position adjustment method of the AR-HUD system may include the following steps:
step 201: and when the virtual image adjustment gesture of the driver is detected, the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system are acquired.
Step 202: and when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture.
Step 203: and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In this application embodiment, can adjust AR-HUD system's virtual image position through driver's virtual image adjustment gesture, need not to adjust through hardware switch, improved the convenience of virtual image adjustment, satisfy simultaneously and cause the demand to different virtual image positions because of driver's difference in height and driving habit, improved driver's security and travelling comfort.
In some embodiments, when detecting the virtual image adjustment gesture of the driver, before acquiring the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system, further comprising:
acquiring behavior images of a specified number of drivers through cameras installed on the automobile;
identifying gesture information of the driver from the specified number of behavioral images, the gesture information including at least a gesture movement direction and a gesture position of the driver;
and determining that the virtual image adjustment gesture of the driver is detected when the gesture movement direction and the gesture position accord with the virtual image adjustment gesture.
In some embodiments, obtaining a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture includes:
comparing the gesture information of the virtual image adjustment gesture with stored gesture information of multiple reference gestures;
and when a reference gesture with gesture information matched with the gesture information of the virtual image adjustment gesture exists in the plurality of reference gestures, acquiring the virtual image adjustment corresponding to the reference gesture.
In some embodiments, adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information includes:
acquiring the current position of the virtual image of the AR-HUD system;
determining an adjustment distance according to the gesture movement distance included in the gesture information;
and adjusting a curved mirror in the AR-HUD system based on the current position of the virtual image of the AR-HUD system according to the moving direction and the adjusting distance carried in the virtual image adjusting instruction so as to adjust the virtual image position of the AR-HUD system, wherein the curved mirror is used for projecting the virtual image of the AR-HUD system.
In some embodiments, obtaining the current location of the virtual image of the AR-HUD system includes:
acquiring the position and projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
All the above optional technical solutions may be combined according to any choice to form an optional embodiment of the present application, which is not described in detail herein.
Fig. 3 is a flowchart of a virtual image position adjustment method of an AR-HUD system according to an embodiment of the present application, where the embodiment is illustrated by applying the virtual image position adjustment method of the AR-HUD system to the application, the virtual image position adjustment method of the AR-HUD system may include the following steps:
step 301: the vehicle acquires the behavior of the driver.
Since the virtual image position of the AR-HUD system may be adjusted by some actions if the driver perceives that the virtual image position of the AR-HUD system is not suitable while driving the vehicle, the vehicle can acquire the behavior action of the driver in order to avoid missing the adjustment action of the driver.
As one example, the operation of the automobile to obtain the behavioral actions of the driver can include: acquiring behavior images of a specified number of drivers through cameras installed on an automobile; identifying gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver; and determining that the virtual image adjustment gesture of the driver is detected when the gesture movement direction and the gesture position accord with the virtual image adjustment gesture.
Note that the specified number can be set in advance according to the need, for example, the specified number can be 3 frames, 5 frames, or the like.
Because the driver performs a dynamic process when adjusting the position of the virtual image of the AR-HUD system through the gesture, in order to avoid missing the gesture, the vehicle needs to acquire a certain number of combined behavior images and perform image recognition from the specified number of behavior images, so as to recognize the gesture movement direction and the gesture position of the driver.
Because the track of the virtual image adjustment gesture is coherent and has a certain rule, the automobile can determine whether the gesture motion is coherent according to the gesture position of the gesture motion of the driver, and determine whether the gesture motion direction and the gesture position of the driver accord with the virtual image adjustment gesture when the gesture motion of the driver is coherent and the gesture motion direction accords with the adjustment track according to whether the gesture motion direction accords with the adjustment track.
As an example, the vehicle can acquire the behavior of the driver once immediately after ignition, can acquire the behavior of the driver when receiving the acquisition instruction, can acquire the behavior of the driver in real time, and can acquire the behavior of the driver after starting the AR-HUD system.
The acquisition instruction can be triggered when the driver acts on the virtual switch or the hardware switch of the automobile through a specified operation, and the specified operation can be a click operation, a sliding operation, a toggle operation, a pressing operation, a voice operation, or the like.
Step 302: when a virtual image adjustment gesture of the driver is detected, the automobile acquires a behavior state of the driver and a display state of the AR-HUD system.
Since the position of the virtual image of the AR-HUD system is related to the eye position of the driver and the display state of the current AR-HUD system, and since some gesture actions of the driver may be unintentionally performed in some cases, these gesture actions exactly match the virtual image adjustment gesture, in order to avoid a mis-adjustment, the car needs to acquire the behavior state of the driver and the display state of the AR-HUD system when detecting the virtual image adjustment gesture of the driver.
The behavior state of the driver includes the eyeball position of the driver.
Since the behavior image of the driver includes the eyeball situation of the driver, the automobile can acquire the eyeball position of the driver from the acquired behavior image, and of course, in order to acquire the accuracy of the information, the automobile can acquire the behavior image of the driver again through the camera and identify the eyeball position of the driver from the acquired behavior image again.
The eyeball position of the driver can be represented by a binocular central position, and the binocular central position can be represented by a three-dimensional coordinate system.
In some embodiments, the vehicle, after acquiring the behavior state of the driver and the display state of the AR-HUD system, is also able to determine whether the behavior state of the driver is in a specified state and whether the display state of the AR-HUD system is the display state of the virtual image position adjustment interface.
The specified state refers to whether or not both eyes of the driver look straight at the front windshield, that is, whether or not both eyes of the driver look straight at the virtual image of the AR-HUD system.
Step 303: and when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is the virtual image position adjustment interface display state, acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture.
When the behavior state of the driver is in the specified state and the display state of the AR-HUD system is the virtual image position adjustment interface display state, the driver is required to adjust the virtual image position of the AR-HUD system currently, so that the automobile can acquire a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture.
In some embodiments, the operation of the automobile to obtain the virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture includes: comparing the gesture information of the virtual image adjustment gesture with stored gesture information of various reference gestures; and when the gesture information in the multiple reference gestures is matched with the gesture information of the virtual image adjustment gesture, acquiring the virtual image adjustment corresponding to the reference gesture.
It should be noted that, the gesture information of the various reference gestures is stored in a gesture library file, and the gesture library file can store the gesture information of the various reference gestures in the form of a photo, for example, an upward gesture action photo, a downward gesture action photo, a left gesture action photo, a right gesture action photo, different modeling hands, small hands, fat hands, thin hands, different gesture modeling pictures, a hand extending out of a middle finger, a hand extending out of a thumb, a hand extending out of a forefinger, a hand extending out of a little finger, left and right hands, and the like, language gestures for various blind persons, and the like.
It should be noted that, the correspondence between the reference gesture and the virtual image adjustment instruction may also be stored in the gesture library file, and therefore, the vehicle may obtain the corresponding virtual image adjustment instruction from the correspondence.
Step 304: and the automobile adjusts the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
As an example, the operation of the automobile to adjust the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information includes: acquiring the current position of the virtual image of the AR-HUD system; determining an adjustment distance according to the gesture movement distance included in the gesture information; and adjusting a curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction and the adjusting distance carried in the virtual image adjusting instruction so as to adjust the virtual image position of the AR-HUD system, wherein the curved mirror is used for projecting the virtual image of the AR-HUD system.
Since the position of the virtual image of the AR-HUD system is adjusted based on the current position of the virtual image of the AR-HUD system, the vehicle needs to acquire the current position of the virtual image of the AR-HUD system.
As an example, the operation of the automobile to obtain the current location of the virtual image of the AR-HUD system includes: acquiring the position and projection angle of a curved mirror in an AR-HUD system, wherein the curved mirror is used for projecting virtual images of the AR-HUD system; and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
Since the AR-HUD virtual image is finally displayed after a series of optical elements are processed and reflected, and finally reflected by a curved mirror (also belonging to the optical elements), and the area of the curved mirror is relatively large and the position is variable, while the fixed position of the element structure in other optical elements is not variable, and some long-term adjustment is easy to fail, in order to avoid failure, the position of the AR-HUD virtual image is usually adjusted by selecting and adjusting the curved mirror, that is, the position and the projection angle of the curved mirror both affect the position of the AR-HUD virtual image. Therefore, in order to determine where the virtual image of the AR-HUD system is currently located, the vehicle needs to determine the position and projection angle of the curved mirror.
In some embodiments, the vehicle is capable of determining a distance between the curved mirror and the front windshield based on the position of the curved mirror, and determining a position of the virtual image of the AR-HUD system on the front windshield based on the distance and the projection angle of the curved mirror.
When the vehicle determines the position of the virtual AR-HUD system image on the front windshield, the position of the virtual AR-HUD system image can be replaced by the position of the reference point in the virtual AR-HUD system image, and the position of the center point in the virtual AR-HUD system image is preferentially selected for the purpose of facilitating adjustment of the position of the virtual AR-HUD system image.
In some embodiments, since the stepper motor is used to adjust the position of the curved mirror in the AR-HUD system, the position of the stepper motor, the position of the curved mirror, and the position of the virtual image of the AR-HUD system are in one-to-one correspondence, the vehicle is also able to determine the position of the virtual image of the AR-HUD system from the position of the stepper motor and, in turn, the reference point position from the position of the virtual image of the AR-HUD system.
It should be noted that, in the embodiment of the present application, the reference point can be represented by three-dimensional coordinates, and the center positions of the eyes and the reference point are located in the same coordinate system.
In some embodiments, the car can determine the gesture movement distance as the adjustment distance, and can also determine N times the gesture movement distance as the adjustment distance, N can be a value set in advance, for example, N is 1, 2, 0.5, and so on.
In some embodiments, the automobile can also determine an adjustment angle according to the eyeball position of the driver in the behavior state, and adjust the curved mirror in the AR-HUD system based on the current position of the virtual image of the AR-HUD system according to the movement direction and the adjustment angle carried in the virtual image adjustment instruction, so as to adjust the virtual image position of the AR-HUD system.
The adjustment angle may be a deviation angle between the reference point position and the eyeball position, and since the eyeball position may be represented by the binocular central position, the deviation angle may be a deviation angle between the reference point position and the binocular central position.
Since the center positions of the eyes and the reference point are located in the same coordinate system, the automobile can determine the deviation angle by the coordinates of the center positions of the eyes and the coordinates of the reference point positions.
In some embodiments, the vehicle is able to determine the deviation angle as the adjustment angle.
In this application embodiment, the car can adjust AR-HUD system's virtual image position through driver's virtual image adjustment gesture, need not to adjust through hardware switch, has improved the convenience of virtual image adjustment, satisfies simultaneously and causes the demand to different virtual image positions because of driver's difference in height and driving habit, has improved driver's security and travelling comfort.
Fig. 4 is a schematic structural diagram of a virtual image position adjustment device of an AR-HUD system according to an embodiment of the present application, where the virtual image position adjustment device of the AR-HUD system may be implemented by software, hardware, or a combination of the two. The virtual image position adjustment device of the AR-HUD system may include: a first acquisition module 401, a second acquisition module 402, and an adjustment module 403.
The first obtaining module 401 is configured to obtain, when a virtual image adjustment gesture of a driver is detected, a behavior state of the driver and a display state of the augmented reality head-up display AR-HUD system;
a second obtaining module 402, configured to obtain a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state;
and the adjusting module 403 is configured to adjust the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In some embodiments, referring to fig. 5, the apparatus further comprises:
a third obtaining module 404, configured to obtain, by using the cameras installed in the automobile, a behavior image of a specified number of drivers;
a recognition module 405, configured to recognize gesture information of the driver from the specified number of behavior images, where the gesture information includes at least a gesture movement direction and a gesture position of the driver;
and the determining module 406 is configured to determine that the virtual image adjustment gesture of the driver is detected when the gesture movement direction and the gesture position conform to the virtual image adjustment gesture.
In some embodiments, referring to fig. 6, the second obtaining module 402 includes:
a comparison submodule 4021, configured to compare the gesture information of the virtual image adjustment gesture with stored gesture information of multiple reference gestures;
the first obtaining submodule 4022 is configured to obtain, when a reference gesture whose gesture information matches with the gesture information of the aerial image adjustment gesture exists in the multiple reference gestures, an aerial image adjustment instruction corresponding to the reference gesture.
In some embodiments, referring to fig. 7, the adjustment module 403 includes:
a second obtaining submodule 4031, configured to obtain a current location of the virtual image of the AR-HUD system;
a determining submodule 4032, configured to determine an adjustment distance according to the gesture movement distance included in the gesture information;
and the adjusting submodule 4033 is configured to adjust a curved mirror in the AR-HUD system based on a current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjusting instruction and the adjusting distance, so as to adjust a virtual image position of the AR-HUD system, where the curved mirror is used for projecting the virtual image of the AR-HUD system.
In some embodiments, the second acquisition sub-module 4031 is configured to:
acquiring the position and projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
In this application embodiment, the car can adjust AR-HUD system's virtual image position through driver's virtual image adjustment gesture, need not to adjust through hardware switch, has improved the convenience of virtual image adjustment, satisfies simultaneously and causes the demand to different virtual image positions because of driver's difference in height and driving habit, has improved driver's security and travelling comfort.
It should be noted that: the virtual image position adjusting device of the AR-HUD system provided in the foregoing embodiment is only exemplified by the division of the foregoing functional modules when adjusting the virtual image position of the AR-HUD system, and in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the virtual image position adjustment device of the AR-HUD system provided in the foregoing embodiment and the virtual image position adjustment method embodiment of the AR-HUD system belong to the same concept, and detailed implementation processes thereof are referred to method embodiments, which are not repeated herein.
The present embodiment also provides a non-transitory computer readable storage medium, which when executed by a processor of a server, enables the server to perform the virtual image position adjustment method of the AR-HUD system provided in the above embodiment.
The embodiments of the present application also provide a computer program product containing instructions, which when run on a server, cause the server to perform the virtual image position adjustment method of the AR-HUD system provided in the above embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The foregoing description of the preferred embodiments is merely illustrative of the present application and is not intended to limit the embodiments of the present application, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the embodiments of the present application are intended to be included within the scope of the present application.
Claims (5)
1. A method for adjusting the position of a virtual image of an AR-HUD system, the method comprising:
acquiring behavior images of a specified number of drivers through cameras installed on an automobile;
identifying gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
when the gesture moving direction and the gesture position accord with the virtual image adjusting gesture, determining that the virtual image adjusting gesture of the driver is detected;
when a virtual image adjustment gesture of a driver is detected, acquiring a behavior state of the driver and a display state of an augmented reality head-up display (AR-HUD) system;
when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture, wherein the behavior state of the driver comprises the eyeball position of the driver, the specified state refers to a binocular direct-view windshield of the driver, and the eyeball position of the driver is acquired from the acquired behavior image of the driver or is identified from the acquired behavior image of the driver;
obtaining the current position of the virtual image of the AR-HUD system comprises the following steps: acquiring the position and projection angle of a curved mirror in an AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system; determining the distance between the curved mirror and the front windshield according to the position of the curved mirror, and determining the current position of the virtual image of the AR-HUD system on the front windshield according to the distance and the projection angle of the curved mirror;
determining an adjustment distance according to the gesture movement distance included in the gesture information;
and adjusting a curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjusting instruction and the adjusting distance so as to adjust the virtual image position of the AR-HUD system.
2. The method of claim 1, wherein the obtaining, according to the gesture information of the aerial image adjustment gesture, an aerial image adjustment instruction corresponding to the aerial image adjustment gesture includes:
comparing the gesture information of the virtual image adjustment gesture with stored gesture information of multiple reference gestures;
and when a reference gesture with gesture information matched with the gesture information of the virtual image adjustment gesture exists in the plurality of reference gestures, acquiring the virtual image adjustment corresponding to the reference gesture.
3. A virtual image position adjustment device of an AR-HUD system, the device comprising:
the third acquisition module is used for acquiring behavior images of a specified number of drivers through cameras installed on the automobile;
the recognition module is used for recognizing gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
the determining module is used for determining that the virtual image adjustment gesture of the driver is detected when the gesture moving direction and the gesture position accord with the virtual image adjustment gesture;
the first acquisition module is used for acquiring the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system when the virtual image adjustment gesture of the driver is detected;
the second obtaining module is configured to obtain a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to gesture information of the virtual image adjustment gesture when the behavior state of the driver is in a specified state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, where the behavior state of the driver includes an eyeball position of the driver, the specified state is a direct-view windshield of both eyes of the driver, and the eyeball position of the driver is obtained from the obtained behavior image of the driver or is identified from the re-obtained behavior image of the driver;
the adjusting module is used for adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information;
the adjustment module includes:
the second obtaining sub-module is configured to obtain a current location of the virtual image of the AR-HUD system, and includes: acquiring the position and projection angle of a curved mirror in an AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system; determining the distance between the curved mirror and the front windshield according to the position of the curved mirror, and determining the current position of the virtual image of the AR-HUD system on the front windshield according to the distance and the projection angle of the curved mirror;
the determining submodule is used for determining an adjustment distance according to the gesture movement distance included in the gesture information;
and the adjusting sub-module is used for adjusting the curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjusting instruction and the adjusting distance so as to adjust the virtual image position of the AR-HUD system.
4. The apparatus of claim 3, wherein the second acquisition module comprises:
the comparison sub-module is used for comparing the gesture information of the virtual image adjustment gesture with stored gesture information of various reference gestures;
the first obtaining sub-module is used for obtaining the virtual image adjusting gesture corresponding to the reference gesture when the reference gesture with gesture information matched with the gesture information of the virtual image adjusting gesture exists in the plurality of reference gestures.
5. A storage medium having stored thereon instructions which, when executed by a processor, implement the steps of the method of any of the preceding claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110332139.6A CN112947761B (en) | 2021-03-26 | 2021-03-26 | Virtual image position adjustment method, device and storage medium of AR-HUD system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110332139.6A CN112947761B (en) | 2021-03-26 | 2021-03-26 | Virtual image position adjustment method, device and storage medium of AR-HUD system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112947761A CN112947761A (en) | 2021-06-11 |
| CN112947761B true CN112947761B (en) | 2023-07-28 |
Family
ID=76227127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110332139.6A Active CN112947761B (en) | 2021-03-26 | 2021-03-26 | Virtual image position adjustment method, device and storage medium of AR-HUD system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112947761B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104875680A (en) * | 2015-06-03 | 2015-09-02 | 深圳市光晕网络科技有限公司 | HUD (head up display) device combining voice and video recognition |
| CN106445104A (en) * | 2016-08-25 | 2017-02-22 | 蔚来汽车有限公司 | Vehicle HUD display system and method |
| WO2017134876A1 (en) * | 2016-02-05 | 2017-08-10 | 日立マクセル株式会社 | Head-up display device and display control method therefor |
| JP2019032362A (en) * | 2017-08-04 | 2019-02-28 | アルパイン株式会社 | Head-up display device and navigation device |
| CN110471532A (en) * | 2019-08-19 | 2019-11-19 | 业成科技（成都）有限公司 | Integrate the car-mounted device of eyeball tracking Yu gesture manipulation |
| WO2019224922A1 (en) * | 2018-05-22 | 2019-11-28 | 三菱電機株式会社 | Head-up display control device, head-up display system, and head-up display control method |
| CN111158491A (en) * | 2019-12-31 | 2020-05-15 | 苏州莱孚斯特电子科技有限公司 | Gesture recognition man-machine interaction method applied to vehicle-mounted HUD |
| JP2020095155A (en) * | 2018-12-12 | 2020-06-18 | 株式会社デンソー | Virtual image display system, display control device and display control program |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10013053B2 (en) * | 2012-01-04 | 2018-07-03 | Tobii Ab | System for gaze interaction |
| KR101438615B1 (en) * | 2012-12-18 | 2014-11-03 | 현대자동차 주식회사 | System and method for providing a user interface using 2 dimension camera in a vehicle |
| KR20140080300A (en) * | 2012-12-20 | 2014-06-30 | 현대자동차주식회사 | Control system for vehicle using hand gesture |
| CN105786361A (en) * | 2014-12-23 | 2016-07-20 | 北京中博安达知识产权服务有限公司 | 3D vehicle-mounted terminal man-machine interaction system |
| CN204821176U (en) * | 2015-07-17 | 2015-12-02 | 樊强 | But human -computer interaction's new line display |
| CN106353886A (en) * | 2016-10-12 | 2017-01-25 | 惠州市华阳多媒体电子有限公司 | Intelligent head up display of automobile |
| CN109445568A (en) * | 2018-08-30 | 2019-03-08 | 丝路视觉科技股份有限公司 | Projection objects control method, device and host |
| KR20200044212A (en) * | 2018-10-11 | 2020-04-29 | 현대자동차주식회사 | Apparatus and method for controlling display of vehicle |
| CN109649275B (en) * | 2018-11-29 | 2020-03-20 | 福瑞泰克智能系统有限公司 | Driving assistance system and method based on augmented reality |
| JP2020126098A (en) * | 2019-02-01 | 2020-08-20 | 株式会社Ｊｖｃケンウッド | Display device and method of installing the same |
| CN111267616A (en) * | 2020-02-28 | 2020-06-12 | 华域视觉科技(上海)有限公司 | Vehicle-mounted head-up display module and method and vehicle |
| CN111443490B (en) * | 2020-04-15 | 2022-11-18 | 杭州赶梦科技有限公司 | Virtual image display area adjusting method of AR HUD |
| CN111638786B (en) * | 2020-05-29 | 2023-07-07 | 电装天研究开发(天津)有限公司 | Display control method, device, equipment and storage medium of vehicle-mounted rear projection display system |
| CN112162688A (en) * | 2020-08-20 | 2021-01-01 | 江苏大学 | Vehicle-mounted virtual screen interactive information system based on gesture recognition |
| CN112241204B (en) * | 2020-12-17 | 2021-08-27 | 宁波均联智行科技股份有限公司 | Gesture interaction method and system of vehicle-mounted AR-HUD |
-
2021
- 2021-03-26 CN CN202110332139.6A patent/CN112947761B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104875680A (en) * | 2015-06-03 | 2015-09-02 | 深圳市光晕网络科技有限公司 | HUD (head up display) device combining voice and video recognition |
| WO2017134876A1 (en) * | 2016-02-05 | 2017-08-10 | 日立マクセル株式会社 | Head-up display device and display control method therefor |
| CN106445104A (en) * | 2016-08-25 | 2017-02-22 | 蔚来汽车有限公司 | Vehicle HUD display system and method |
| JP2019032362A (en) * | 2017-08-04 | 2019-02-28 | アルパイン株式会社 | Head-up display device and navigation device |
| WO2019224922A1 (en) * | 2018-05-22 | 2019-11-28 | 三菱電機株式会社 | Head-up display control device, head-up display system, and head-up display control method |
| JP2020095155A (en) * | 2018-12-12 | 2020-06-18 | 株式会社デンソー | Virtual image display system, display control device and display control program |
| CN110471532A (en) * | 2019-08-19 | 2019-11-19 | 业成科技（成都）有限公司 | Integrate the car-mounted device of eyeball tracking Yu gesture manipulation |
| CN111158491A (en) * | 2019-12-31 | 2020-05-15 | 苏州莱孚斯特电子科技有限公司 | Gesture recognition man-machine interaction method applied to vehicle-mounted HUD |
Non-Patent Citations (5)
| Title |
|---|
| AR-HUD system for tower crane on construction field;KyeongHoon Park等;2011 IEEE International Symposium on VR Innovation;全文 * |
| 抬头显示系统在汽车上的应用;杨飞;林绍华;吴春建;;轻型汽车技术(第Z4期);全文 * |
| 抬头显示系统成像参数测量技术现状分析;鲁宇;张靖;冉舒文;邹耀;郑立兴;陈伟民;;激光杂志(第02期);全文 * |
| 某车载抬头显示光学系统设计研究;陈璐玲;;中国新技术新产品(第13期);全文 * |
| 汽车手势交互设计研究;景春晖;张晶;邓椿山;;包装工程(第08期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112947761A (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10913463B2 (en) | Gesture based control of autonomous vehicles | |
| US9946078B2 (en) | Head-up display device | |
| KR101921969B1 (en) | augmented reality head-up display apparatus and method for vehicles | |
| CN112750206A (en) | Augmented reality wearable system for vehicle occupants | |
| US9589469B2 (en) | Display control method, display control apparatus, and display apparatus | |
| US20180113505A1 (en) | Virtual reality interaction method, apparatus and system | |
| JP6836206B2 (en) | Display control device and display control program | |
| JP2005138755A (en) | Device and program for displaying virtual images | |
| CN111252074B (en) | Multi-modal control method, device, computer-readable storage medium and vehicle | |
| JP6620977B2 (en) | Display control device, projection device, and display control program | |
| CN112162688A (en) | Vehicle-mounted virtual screen interactive information system based on gesture recognition | |
| CN112558305A (en) | Control method, device and medium for display picture and head-up display control system | |
| CN113064279B (en) | Virtual image position adjusting method, device and storage medium of AR-HUD system | |
| CN107472140B (en) | Eye-control vehicle-mounted HUD system | |
| JPWO2020105685A1 (en) | Display controls, methods, and computer programs | |
| CN112947761B (en) | Virtual image position adjustment method, device and storage medium of AR-HUD system | |
| US20210300183A1 (en) | In-vehicle display apparatus, method for controlling in-vehicle display apparatus, and computer program | |
| EP2945139A1 (en) | Display control device, display control method, display control program, and projection device | |
| CN113002461A (en) | Virtual image position adjusting method, device and storage medium of AR-HUD system | |
| CN115223231A (en) | Sight direction detection method and device | |
| CN116501167A (en) | In-vehicle interaction system based on gesture operation and vehicle | |
| US20190161093A1 (en) | Driving assist device | |
| CN113525402B (en) | Advanced assisted driving and unmanned visual field intelligent response method and system | |
| EP4029716A1 (en) | Vehicle interactive system and method, storage medium, and vehicle | |
| CN114312569A (en) | Method and device for managing projection operation in vehicle and vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |