CN112947761A - Virtual image position adjusting method, device and storage medium of AR-HUD system - Google Patents
Virtual image position adjusting method, device and storage medium of AR-HUD system Download PDFInfo
- Publication number
- CN112947761A CN112947761A CN202110332139.6A CN202110332139A CN112947761A CN 112947761 A CN112947761 A CN 112947761A CN 202110332139 A CN202110332139 A CN 202110332139A CN 112947761 A CN112947761 A CN 112947761A
- Authority
- CN
- China
- Prior art keywords
- virtual image
- gesture
- hud system
- adjusting
- driver
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0149—Head-up displays characterised by mechanical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0149—Head-up displays characterised by mechanical features
- G02B2027/0154—Head-up displays characterised by mechanical features with movable elements
Abstract
The embodiment of the application discloses a virtual image position adjusting method and device of an AR-HUD system and a storage medium, and belongs to the technical field of intelligent automobile driving. The method comprises the following steps: when a virtual image adjusting gesture of a driver is detected, acquiring a behavior state of the driver and a display state of an AR-HUD system; when the behavior state of the driver is in the designated state and the display state of the AR-HUD system is the virtual image position adjusting interface display state, acquiring a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to gesture information of the virtual image adjusting gesture; and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information. The virtual image position of the AR-HUD system is adjusted through the virtual image adjusting gesture of the driver, so that the requirements for different virtual image positions due to different heights of the driver and different driving habits are met, and the safety and the comfort of the driver are improved.
Description
Technical Field
The embodiment of the application relates to the technical field of intelligent automobile driving, in particular to a virtual image position adjusting method and device of an AR-HUD (Augmented Reality Head Up Display) system and a storage medium.
Background
With the continuous development of society, automobiles become indispensable transportation means in daily life of people. However, as the number of automobiles increases, the frequency of occurrence of traffic accidents is also increasing. In order to improve driving safety, the automobile can be assisted to drive through the AR-HUD system. The AR-HUD system can project some important information that the driver needs when driving to the windscreen before the car, shows that the virtual image fuses with the outdoor scene around the car, and the light that fuses the outdoor scene intakes driver's eyes through the reflection, makes the driver see the virtual image that fuses with the outdoor scene to the instrument need not look over by the low head to the driver, and the place ahead road surface is watched all the time to eyes, has improved because of the influence of low head to safe driving. However, since a driver driving a vehicle may change the position of the virtual image, the position of the virtual image needs to be adjusted because the height of the driver is not uniform and the driving habits are not uniform.
At present, a hardware switch for adjusting the virtual image position can be arranged in the automobile, and a driver adjusts the virtual image position through the hardware switch. However, sometimes, the driver cannot find the hardware switch, so that the driving experience of the driver is poor, and the viscosity of the user is reduced.
Disclosure of Invention
The embodiment of the application provides a virtual image position adjusting method and device of an AR-HUD system and a storage medium, and can solve the problems of driving comfort and low safety caused by inconvenient adjustment of the virtual image position of the AR-HUD system in the related art. The technical scheme is as follows:
in one aspect, a virtual image position adjustment method for an AR-HUD system is provided, the method including:
when a virtual image adjusting gesture of a driver is detected, acquiring a behavior state of the driver and a display state of an augmented reality head-up display (AR-HUD) system;
when the behavior state of the driver is in an appointed state and the display state of the AR-HUD system is a virtual image position adjusting interface display state, acquiring a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to gesture information of the virtual image adjusting gesture;
and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In some embodiments, before acquiring the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system when the virtual image adjustment gesture of the driver is detected, the method further includes:
acquiring behavior images of a specified number of drivers through a camera installed on the automobile;
recognizing gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and when the gesture moving direction and the gesture position accord with a virtual image adjusting gesture, determining that the virtual image adjusting gesture of the driver is detected.
In some embodiments, the obtaining, according to the gesture information of the virtual image adjustment gesture, a virtual image adjustment instruction corresponding to the virtual image adjustment gesture includes:
comparing the gesture information of the virtual image adjusting gesture with stored gesture information of various reference gestures;
and when a reference gesture with gesture information matched with the gesture information of the virtual image adjusting gesture exists in the multiple reference gestures, acquiring a virtual image adjusting instruction corresponding to the reference gesture.
In some embodiments, the adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information includes:
acquiring the current position of the virtual image of the AR-HUD system;
determining an adjustment distance according to a gesture moving distance included in the gesture information;
according to the moving direction that carries in the virtual image adjustment instruction with adjust the distance AR-HUD system virtual image is to on the basis of the current position of AR-HUD system virtual image the curved mirror in the AR-HUD system is adjusted to the adjustment AR-HUD system's virtual image position, the curved mirror is used for the projection AR-HUD system virtual image.
In some embodiments, the acquiring the current position of the virtual image of the AR-HUD system includes:
acquiring the position and the projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
In another aspect, there is provided a virtual image position adjustment apparatus of an AR-HUD system, the apparatus including:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring the behavior state of a driver and the display state of an augmented reality head-up display (AR-HUD) system when a virtual image adjustment gesture of the driver is detected;
the second obtaining module is used for obtaining a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to gesture information of the virtual image adjusting gesture when the behavior state of the driver is in an appointed state and the display state of the AR-HUD system is a virtual image position adjusting interface display state;
and the adjusting module is used for adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In some embodiments, the apparatus further comprises:
the third acquisition module is used for acquiring behavior images of a specified number of drivers through the cameras installed on the automobile;
the recognition module is used for recognizing gesture information of the driver from the specified number of behavior images, and the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and the determining module is used for determining that the virtual image adjusting gesture of the driver is detected when the gesture moving direction and the gesture position accord with the virtual image adjusting gesture.
In some embodiments, the second obtaining module comprises:
the comparison submodule is used for comparing the gesture information of the virtual image adjusting gesture with the stored gesture information of various reference gestures;
and the first obtaining submodule is used for obtaining a virtual image adjusting instruction corresponding to the reference gesture when the reference gesture exists in the multiple reference gestures, wherein the gesture information of the reference gesture is matched with the gesture information of the virtual image adjusting gesture.
In some embodiments, the adjustment module comprises:
the second acquisition submodule is used for acquiring the current position of the virtual image of the AR-HUD system;
the determining submodule is used for determining an adjusting distance according to the gesture moving distance included in the gesture information;
and the adjustment submodule is used for adjusting the curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjustment instruction and the adjustment distance, so as to adjust the virtual image position of the AR-HUD system, wherein the curved mirror is used for projecting the virtual image of the AR-HUD system.
In some embodiments, the second acquisition submodule is to:
acquiring the position and the projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
In another aspect, a computer-readable storage medium is provided, having instructions stored thereon, which when executed by a processor, implement any one of the above-described virtual image position adjustment methods of an AR-HUD system.
The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:
in this application embodiment, can adjust the virtual image position of AR-HUD system through driver's virtual image adjustment gesture, need not to adjust through the hardware switch, improve the convenience of virtual image adjustment, satisfy simultaneously because of driver's the difference of height and the driving habit difference cause the demand to different virtual image positions, improved driver's security and travelling comfort.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic diagram of a virtual image position adjustment system of an AR-HUD system according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a virtual image position adjustment method of an AR-HUD system according to an embodiment of the present disclosure;
FIG. 3 is a flowchart of another virtual image position adjustment method for an AR-HUD system according to an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of a virtual image position adjustment apparatus of an AR-HUD system according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of another virtual image position adjustment apparatus of an AR-HUD system according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a second obtaining module according to an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of an adjustment module according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.
Before explaining a virtual image position adjusting method of an AR-HUD system provided in the embodiment of the present application in detail, an application scenario and a system architecture provided in the embodiment of the present application are explained first.
With the continuous development of society, automobiles become indispensable transportation means in daily life of people, the accompanying traffic safety problem becomes more serious, and traffic accidents become the first killer of lives and properties of people in the current society. Therefore, automobile safety is becoming a major direction for future automobile development. The AR-HUD system is an automobile auxiliary vision safe driving system, and the system projects some important information required by a driver when driving to the windshield glass of an automobile through the light path of the AR-HUD system, so that the driver does not need to look down at the instrument panel, eyes always look at the front field road surface, and the influence of head lowering on safe driving is reduced. Because the driver of driving the car may take place to change, the height of driver is inconsistent, and the driving habit is inconsistent to the position requirement to the virtual image is also different, consequently, needs adjust the virtual image position.
Based on the application scene, the embodiment of the application provides a virtual image position adjusting method of an AR-HUD system, which can improve driving safety and comfort.
Next, a virtual image position adjustment system architecture of the AR-HUD system provided in the embodiment of the present application is explained.
Fig. 1 is a schematic diagram of a virtual image position adjustment system architecture of an AR-HUD system provided in an embodiment of the present application, and referring to fig. 1, the system architecture includes a sound system host 1, a camera 2, a library file module 3, an optical machine controller 4, a stepping motor 5, and a curved mirror 6. The sound equipment main machine 1 is respectively connected with the camera 2, the library file module 3 and the optical machine controller 4, the optical machine controller 4 is also connected with the stepping motor 5, and the stepping motor 5 is connected with the curved mirror 6.
As an example, the camera 2 is configured to acquire a virtual adjustment gesture, a behavior state, and the like of the driver, and to send the acquired virtual adjustment gesture and behavior state of the driver to the main audio body 1. The sound equipment host 1 is used for receiving the virtual adjustment gesture and the behavior state sent by the camera 2, acquiring the display state of the AR-HUD system from the optical machine controller 4, acquiring a virtual image adjustment instruction corresponding to the virtual image adjustment gesture from the library file module 4 according to gesture information of the virtual image adjustment gesture when the behavior state of a driver is in an appointed state and the display state of the AR-HUD system is a virtual image position adjustment interface display state, acquiring an adjustment position angle and an adjustment distance according to the virtual image adjustment instruction, the gesture information and the behavior state of the driver, and sending the adjustment position angle and the adjustment distance to the optical machine controller 4; the optical machine controller 4 sends the virtual image adjusting instruction, the adjusting position angle and the adjusting distance to the stepping motor 5; the stepping motor 5 is used for adjusting the curved mirror 6 according to the virtual adjustment instruction, the adjustment position angle and the adjustment distance so as to adjust the virtual image position of the AR-HUD system.
It should be understood by those skilled in the art that the foregoing system architecture is merely exemplary, and other modules, components, etc. that exist or may come into existence in the future, as applicable to the present application, are intended to be included within the scope of the present application and are incorporated herein by reference.
Fig. 2 is a flowchart of a virtual image position adjustment method of an AR-HUD system according to an embodiment of the present disclosure, where the virtual image position adjustment method of the AR-HUD system may include the following steps:
step 201: when a virtual image adjusting gesture of the driver is detected, the behavior state of the driver and the display state of the AR-HUD system are obtained.
Step 202: and when the behavior state of the driver is in an appointed state and the display state of the AR-HUD system is a virtual image position adjusting interface display state, acquiring a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to the gesture information of the virtual image adjusting gesture.
Step 203: and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In this application embodiment, can adjust the virtual image position of AR-HUD system through driver's virtual image adjustment gesture, need not to adjust through the hardware switch, improve the convenience of virtual image adjustment, satisfy simultaneously because of driver's the difference of height and the driving habit difference cause the demand to different virtual image positions, improved driver's security and travelling comfort.
In some embodiments, when the virtual image adjustment gesture of the driver is detected, before acquiring the behavior state of the driver and the display state of the augmented reality head-up display AR-HUD system, the method further includes:
acquiring behavior images of a specified number of drivers through a camera installed on the automobile;
recognizing gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and when the gesture moving direction and the gesture position accord with a virtual image adjusting gesture, determining that the virtual image adjusting gesture of the driver is detected.
In some embodiments, obtaining a virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture includes:
comparing the gesture information of the virtual image adjusting gesture with stored gesture information of various reference gestures;
and when the reference gesture with the gesture information matched with the gesture information of the virtual image adjusting gesture exists in the multiple reference gestures, acquiring a virtual image adjusting instruction corresponding to the reference gesture.
In some embodiments, adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information includes:
acquiring the current position of the virtual image of the AR-HUD system;
determining an adjustment distance according to the gesture moving distance included in the gesture information;
according to the moving direction and the adjustment distance that carry in this virtual image adjustment instruction, adjust the curved mirror in this AR-HUD system on the position basis that this AR-HUD system virtual image is present to the virtual image position of this AR-HUD system is adjusted, this curved mirror is used for projecting this AR-HUD system virtual image.
In some embodiments, obtaining the current position of the virtual image of the AR-HUD system comprises:
acquiring the position and the projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
All the above optional technical solutions can be combined arbitrarily to form an optional embodiment of the present application, and the present application embodiment is not described in detail again.
Fig. 3 is a flowchart of a virtual image position adjusting method of an AR-HUD system according to an embodiment of the present application, which is exemplified in an application of the virtual image position adjusting method of the AR-HUD system in this embodiment, where the virtual image position adjusting method of the AR-HUD system may include the following steps:
step 301: the automobile acquires the behavior action of the driver.
Since the virtual image position of the AR-HUD system may be adjusted by some actions if the driver perceives that the virtual image position of the AR-HUD system is not appropriate when driving the vehicle, the vehicle can acquire the behavior action of the driver in order to avoid missing the adjustment action of the driver.
As an example, the operation of the automobile to acquire the behavior action of the driver can include: acquiring behavior images of a specified number of drivers through a camera mounted on an automobile; recognizing gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver; and when the gesture moving direction and the gesture position accord with the virtual image adjusting gesture, determining that the virtual image adjusting gesture of the driver is detected.
It should be noted that the specified number can be set in advance according to the requirement, for example, the specified number can be 3 frames, 5 frames, and so on.
Because the gesture action is a dynamic process when the driver adjusts the position of the virtual image of the AR-HUD system through the gesture action, in order to avoid missing the gesture action, the automobile needs to acquire and combine a certain number of behavior images and perform image recognition from the specified number of behavior images, so that the gesture moving direction and the gesture position of the driver are recognized.
Because the track of virtual image adjustment gesture is coherent and has certain law, therefore, the car can confirm whether the gesture action is coherent according to the gesture position of driver's gesture action, whether accord with the adjustment track according to the gesture moving direction, when driver's gesture action is coherent, and the gesture moving direction accords with the adjustment track, confirms that driver's gesture moving direction and gesture position accord with the virtual image adjustment gesture.
As an example, the automobile can acquire the behavior of the driver once immediately after ignition, acquire the behavior of the driver when receiving the acquisition instruction, acquire the behavior of the driver in real time, and acquire the behavior of the driver after starting the AR-HUD system.
It should be noted that the acquisition instruction can be triggered when the driver acts on a virtual switch or a hardware switch of the automobile through a specified operation, and the specified operation can be a click operation, a sliding operation, a toggle operation, a push operation, a voice operation, and the like.
Step 302: when the virtual image adjustment gesture of the driver is detected, the automobile acquires the behavior state of the driver and the display state of the AR-HUD system.
Since the position of the virtual image of the AR-HUD system is related to the position of the eyes of the driver and the display state of the current AR-HUD system, and since some gesture actions of the driver may be performed unintentionally, these gesture actions are matched with the virtual image adjustment gesture exactly, so to avoid misadjustment, the automobile needs to acquire the behavior state of the driver and the display state of the AR-HUD system when detecting the virtual image adjustment gesture of the driver.
It should be noted that the behavior state of the driver includes the eyeball position of the driver.
Since the eyeball condition of the driver is included in the behavior image of the driver, the automobile can acquire the eyeball position of the driver from the acquired behavior image, and of course, for the accuracy of information acquisition, the automobile can also acquire the behavior image of the driver again through the camera and identify the eyeball position of the driver from the acquired behavior image.
The position of the eyeball of the driver can be represented by the center position of both eyes, and the center position of both eyes can be represented by a three-dimensional coordinate system.
In some embodiments, after acquiring the behavior state of the driver and the display state of the AR-HUD system, the automobile can further determine whether the behavior state of the driver is in a specified state, and whether the display state of the AR-HUD system is the display state of the virtual image position adjustment interface.
Note that the specified state is whether or not the driver's eyes are looking directly at the front windshield, that is, whether or not the driver's eyes are looking directly at the virtual image of the AR-HUD system.
Step 303: and when the behavior state of the driver is in the designated state and the display state of the AR-HUD system is the virtual image position adjusting interface display state, acquiring a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to the gesture information of the virtual image adjusting gesture.
Because when the behavior state of the driver is in the designated state and the display state of the AR-HUD system is the virtual image position adjusting interface display state, it indicates that the driver needs to adjust the virtual image position of the AR-HUD system currently, therefore, the automobile can acquire the virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to the gesture information of the virtual image adjusting gesture.
In some embodiments, the operation of the automobile obtaining the virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture includes: comparing the gesture information of the virtual image adjusting gesture with stored gesture information of various reference gestures; and when the reference gesture with the gesture information matched with the gesture information of the virtual image adjusting gesture exists in the multiple reference gestures, acquiring a virtual image adjusting instruction corresponding to the reference gesture.
It should be noted that the gesture information of the multiple reference gestures is stored in a gesture library file, and the gesture library file can store the gesture information of the multiple reference gestures in the form of photos, such as an upward gesture motion photo, a downward gesture motion photo, a leftward gesture motion picture, a rightward gesture motion photo, different-shaped hands, small hands, fat hands, thin hands, different-shaped pictures, a hand stretching out a middle finger, a hand stretching out a thumb, a hand stretching out a forefinger, a hand stretching out a small thumb, a left hand and a right hand, and the like, and various language gestures for blind people.
It should be further noted that the gesture library file may further store a corresponding relationship between the reference gesture and the virtual image adjustment instruction, so that the automobile may obtain the corresponding virtual image adjustment instruction from the corresponding relationship.
Step 304: and the automobile adjusts the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
As an example, the operation of the automobile to adjust the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information includes: acquiring the current position of a virtual image of the AR-HUD system; determining an adjustment distance according to a gesture moving distance included in the gesture information; according to the moving direction and the adjusting distance carried in the virtual image adjusting instruction, the curved mirror in the AR-HUD system is adjusted on the basis of the current position of the virtual image of the AR-HUD system to adjust the virtual image position of the AR-HUD system, and the curved mirror is used for projecting the virtual image of the AR-HUD system.
Because the position of the virtual image of the AR-HUD system is adjusted based on the current position of the virtual image of the AR-HUD system, the automobile needs to acquire the current position of the virtual image of the AR-HUD system.
As an example, the operation of the automobile for acquiring the current position of the virtual image of the AR-HUD system comprises the following steps: acquiring the position and the projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system; and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
Because the AR-HUD virtual image is finally displayed after being reflected by the curved mirror (also belonging to an optical element) after a series of optical elements are processed, reflected and the like, the area of the curved mirror is relatively large and the position of the curved mirror can be changed, while the fixed positions of some element structures in other optical elements can not be changed, and some long-term adjustment is easy to fail, in order to avoid failure, the curved mirror is usually selected to be adjusted to adjust the position of the AR-HUD virtual image, that is, the position and the projection angle of the curved mirror can influence the position of the AR-HUD virtual image. Therefore, in order to determine where the virtual image of the AR-HUD system is currently located, the automobile needs to determine the position and projection angle of the curved mirror.
In some embodiments, the vehicle can determine the distance between the curved mirror and the front windshield based on the position of the curved mirror, and determine the position of the virtual image of the AR-HUD system on the front windshield based on the distance and the projection angle of the curved mirror.
It should be noted that, when the automobile determines the position of the virtual image of the AR-HUD system on the front windshield, the position representing the virtual image of the AR-HUD system can be replaced by the position of the reference point in the virtual image of the AR-HUD system, and in order to facilitate adjustment of the position of the virtual image of the AR-HUD system, the position of the center point in the virtual image of the AR-HUD system is preferentially selected by the position of the reference point.
In some embodiments, since the stepping motor is used to adjust the position of the curved mirror in the AR-HUD system, the position of the stepping motor, the position of the curved mirror, and the position of the virtual image of the AR-HUD system are in one-to-one correspondence, the automobile can also determine the position of the virtual image of the AR-HUD system according to the position of the stepping motor, and then determine the position of the reference point according to the position of the virtual image of the AR-HUD system.
It should be noted that, in the embodiment of the present application, the reference point position can be represented by three-dimensional coordinates, and the center position of both eyes and the reference point position are located in the same coordinate system.
In some embodiments, the car can determine the gesture movement distance as the adjustment distance, and can also determine N times the gesture movement distance as the adjustment distance, where N can be a preset value, for example, N is 1, 2, 0.5, and so on.
In some embodiments, the automobile can also determine an adjustment angle according to the eyeball position of the driver in the behavior state, and adjust the curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction and the adjustment angle carried in the virtual image adjustment instruction, so as to adjust the virtual image position of the AR-HUD system.
The adjustment angle may be a deviation angle between the reference point position and the eyeball position, and the eyeball position may be indicated by the center position of both eyes, and therefore the deviation angle may be a deviation angle between the reference point position and the center position of both eyes.
Because the center positions of the two eyes and the position of the reference point are located in the same coordinate system, the automobile can determine the deviation angle through the coordinate of the center positions of the two eyes and the coordinate of the position of the reference point.
In some embodiments, the vehicle can determine the deviation angle as the adjustment angle.
In this application embodiment, the car can adjust the virtual image position of AR-HUD system through driver's virtual image adjustment gesture, need not to adjust through the hardware switch, has improved the convenience of virtual image adjustment, satisfies simultaneously because of driver's the difference of height and the driving habit difference and causes the demand to different virtual image positions, has improved driver's security and travelling comfort.
Fig. 4 is a schematic structural diagram of a virtual image position adjustment device of an AR-HUD system according to an embodiment of the present application, where the virtual image position adjustment device of the AR-HUD system may be implemented by software, hardware, or a combination of the two. The virtual image position adjustment device of the AR-HUD system may include: a first obtaining module 401, a second obtaining module 402 and an adjusting module 403.
A first obtaining module 401, configured to obtain a behavior state of a driver and a display state of an augmented reality head-up display AR-HUD system when a virtual image adjustment gesture of the driver is detected;
a second obtaining module 402, configured to obtain, according to gesture information of the virtual image adjustment gesture, a virtual image adjustment instruction corresponding to the virtual image adjustment gesture when the behavior state of the driver is in an assigned state and a display state of the AR-HUD system is a virtual image position adjustment interface display state;
an adjusting module 403, configured to adjust a position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
In some embodiments, referring to fig. 5, the apparatus further comprises:
a third obtaining module 404, configured to obtain behavior images of a specified number of drivers through a camera mounted on the vehicle;
a recognition module 405, configured to recognize gesture information of the driver from the specified number of behavior images, where the gesture information at least includes a gesture moving direction and a gesture position of the driver;
a determining module 406, configured to determine that a virtual image adjustment gesture of the driver is detected when the gesture moving direction and the gesture position conform to the virtual image adjustment gesture.
In some embodiments, referring to fig. 6, the second obtaining module 402 comprises:
the comparison submodule 4021 is configured to compare the gesture information of the virtual image adjustment gesture with stored gesture information of multiple reference gestures;
the first obtaining submodule 4022 is configured to obtain a virtual image adjustment instruction corresponding to the reference gesture when the reference gesture exists in the multiple reference gestures, where the gesture information of the multiple reference gestures is matched with the gesture information of the virtual image adjustment gesture.
In some embodiments, referring to fig. 7, the adjustment module 403 includes:
a second obtaining submodule 4031, configured to obtain a current position of a virtual image of the AR-HUD system;
the determining submodule 4032 is configured to determine an adjustment distance according to the gesture movement distance included in the gesture information;
and the adjusting submodule 4033 is used for adjusting a curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjusting instruction and the adjusting distance, so as to adjust the virtual image position of the AR-HUD system, wherein the curved mirror is used for projecting the virtual image of the AR-HUD system.
In some embodiments, the second obtaining sub-module 4031 is configured to:
acquiring the position and the projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
In this application embodiment, the car can adjust the virtual image position of AR-HUD system through driver's virtual image adjustment gesture, need not to adjust through the hardware switch, has improved the convenience of virtual image adjustment, satisfies simultaneously because of driver's the difference of height and the driving habit difference and causes the demand to different virtual image positions, has improved driver's security and travelling comfort.
It should be noted that: the virtual image position adjusting device of the AR-HUD system provided in the above embodiment is illustrated by only dividing the above functional modules when adjusting the virtual image position of the AR-HUD system, and in practical applications, the above function allocation may be completed by different functional modules as needed, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the virtual image position adjusting device of the AR-HUD system provided in the above embodiment and the virtual image position adjusting method embodiment of the AR-HUD system belong to the same concept, and specific implementation processes thereof are described in detail in the method embodiment and are not described herein again.
An embodiment of the present application further provides a non-transitory computer-readable storage medium, where instructions in the storage medium, when executed by a processor of a server, enable the server to execute the virtual image position adjustment method of the AR-HUD system provided in the foregoing embodiment.
The embodiment of the present application further provides a computer program product containing instructions, which when run on a server, causes the server to execute the virtual image position adjustment method of the AR-HUD system provided in the above embodiment.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A virtual image position adjustment method for an AR-HUD system, the method comprising:
when a virtual image adjusting gesture of a driver is detected, acquiring a behavior state of the driver and a display state of an augmented reality head-up display (AR-HUD) system;
when the behavior state of the driver is in an appointed state and the display state of the AR-HUD system is a virtual image position adjusting interface display state, acquiring a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to gesture information of the virtual image adjusting gesture;
and adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
2. The method of claim 1, wherein prior to acquiring the behavioral state of the driver and the display state of the augmented reality heads-up display (AR-HUD) system when the virtual image adjustment gesture of the driver is detected, further comprising:
acquiring behavior images of a specified number of drivers through a camera installed on the automobile;
recognizing gesture information of the driver from the specified number of behavior images, wherein the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and when the gesture moving direction and the gesture position accord with a virtual image adjusting gesture, determining that the virtual image adjusting gesture of the driver is detected.
3. The method as claimed in claim 1, wherein the obtaining of the virtual image adjustment instruction corresponding to the virtual image adjustment gesture according to the gesture information of the virtual image adjustment gesture includes:
comparing the gesture information of the virtual image adjusting gesture with stored gesture information of various reference gestures;
and when a reference gesture with gesture information matched with the gesture information of the virtual image adjusting gesture exists in the multiple reference gestures, acquiring a virtual image adjusting instruction corresponding to the reference gesture.
4. The method of claim 1, wherein the adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjustment instruction and the gesture information comprises:
acquiring the current position of the virtual image of the AR-HUD system;
determining an adjustment distance according to a gesture moving distance included in the gesture information;
according to the moving direction that carries in the virtual image adjustment instruction with adjust the distance AR-HUD system virtual image is to on the basis of the current position of AR-HUD system virtual image the curved mirror in the AR-HUD system is adjusted to the adjustment AR-HUD system's virtual image position, the curved mirror is used for the projection AR-HUD system virtual image.
5. The method of claim 4, wherein said obtaining a current location of a virtual image of said AR-HUD system comprises:
acquiring the position and the projection angle of a curved mirror in the AR-HUD system, wherein the curved mirror is used for projecting a virtual image of the AR-HUD system;
and determining the current position of the virtual image of the AR-HUD system according to the position and the projection angle of the curved mirror.
6. A virtual image position adjustment device of an AR-HUD system, characterized by comprising:
the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring the behavior state of a driver and the display state of an augmented reality head-up display (AR-HUD) system when a virtual image adjustment gesture of the driver is detected;
the second obtaining module is used for obtaining a virtual image adjusting instruction corresponding to the virtual image adjusting gesture according to gesture information of the virtual image adjusting gesture when the behavior state of the driver is in an appointed state and the display state of the AR-HUD system is a virtual image position adjusting interface display state;
and the adjusting module is used for adjusting the position of the virtual image of the AR-HUD system according to the virtual image adjusting instruction and the gesture information.
7. The apparatus of claim 6, wherein the apparatus further comprises:
the third acquisition module is used for acquiring behavior images of a specified number of drivers through the cameras installed on the automobile;
the recognition module is used for recognizing gesture information of the driver from the specified number of behavior images, and the gesture information at least comprises a gesture moving direction and a gesture position of the driver;
and the determining module is used for determining that the virtual image adjusting gesture of the driver is detected when the gesture moving direction and the gesture position accord with the virtual image adjusting gesture.
8. The apparatus of claim 6, wherein the second obtaining module comprises:
the comparison submodule is used for comparing the gesture information of the virtual image adjusting gesture with the stored gesture information of various reference gestures;
and the first obtaining submodule is used for obtaining a virtual image adjusting instruction corresponding to the reference gesture when the reference gesture exists in the multiple reference gestures, wherein the gesture information of the reference gesture is matched with the gesture information of the virtual image adjusting gesture.
9. The apparatus of claim 6, wherein the adjustment module comprises:
the second acquisition submodule is used for acquiring the current position of the virtual image of the AR-HUD system;
the determining submodule is used for determining an adjusting distance according to the gesture moving distance included in the gesture information;
and the adjustment submodule is used for adjusting the curved mirror in the AR-HUD system on the basis of the current position of the virtual image of the AR-HUD system according to the moving direction carried in the virtual image adjustment instruction and the adjustment distance, so as to adjust the virtual image position of the AR-HUD system, wherein the curved mirror is used for projecting the virtual image of the AR-HUD system.
10. A storage medium having stored thereon instructions which, when executed by a processor, carry out the steps of the method of any of the preceding claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110332139.6A CN112947761B (en) | 2021-03-26 | 2021-03-26 | Virtual image position adjustment method, device and storage medium of AR-HUD system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110332139.6A CN112947761B (en) | 2021-03-26 | 2021-03-26 | Virtual image position adjustment method, device and storage medium of AR-HUD system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112947761A true CN112947761A (en) | 2021-06-11 |
| CN112947761B CN112947761B (en) | 2023-07-28 |
Family
ID=76227127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110332139.6A Active CN112947761B (en) | 2021-03-26 | 2021-03-26 | Virtual image position adjustment method, device and storage medium of AR-HUD system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112947761B (en) |
Citations (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103869970A (en) * | 2012-12-18 | 2014-06-18 | 现代自动车株式会社 | System and method for manipulating user interface by 2d camera |
| CN103885580A (en) * | 2012-12-20 | 2014-06-25 | 现代自动车株式会社 | Control system for using hand gesture for vehicle |
| CN104145232A (en) * | 2012-01-04 | 2014-11-12 | 托比技术股份公司 | System for gaze interaction |
| CN104875680A (en) * | 2015-06-03 | 2015-09-02 | 深圳市光晕网络科技有限公司 | HUD (head up display) device combining voice and video recognition |
| CN204821176U (en) * | 2015-07-17 | 2015-12-02 | 樊强 | But human -computer interaction's new line display |
| CN105786361A (en) * | 2014-12-23 | 2016-07-20 | 北京中博安达知识产权服务有限公司 | 3D vehicle-mounted terminal man-machine interaction system |
| CN106353886A (en) * | 2016-10-12 | 2017-01-25 | 惠州市华阳多媒体电子有限公司 | Intelligent head up display of automobile |
| CN106445104A (en) * | 2016-08-25 | 2017-02-22 | 蔚来汽车有限公司 | Vehicle HUD display system and method |
| WO2017134876A1 (en) * | 2016-02-05 | 2017-08-10 | 日立マクセル株式会社 | Head-up display device and display control method therefor |
| JP2019032362A (en) * | 2017-08-04 | 2019-02-28 | アルパイン株式会社 | Head-up display device and navigation device |
| CN109445568A (en) * | 2018-08-30 | 2019-03-08 | 丝路视觉科技股份有限公司 | Projection objects control method, device and host |
| CN109649275A (en) * | 2018-11-29 | 2019-04-19 | 福瑞泰克智能系统有限公司 | A kind of driving assistance system and method based on augmented reality |
| CN110471532A (en) * | 2019-08-19 | 2019-11-19 | 业成科技（成都）有限公司 | Integrate the car-mounted device of eyeball tracking Yu gesture manipulation |
| WO2019224922A1 (en) * | 2018-05-22 | 2019-11-28 | 三菱電機株式会社 | Head-up display control device, head-up display system, and head-up display control method |
| CN111038402A (en) * | 2018-10-11 | 2020-04-21 | 现代自动车株式会社 | Apparatus and method for controlling display of vehicle |
| CN111158491A (en) * | 2019-12-31 | 2020-05-15 | 苏州莱孚斯特电子科技有限公司 | Gesture recognition man-machine interaction method applied to vehicle-mounted HUD |
| CN111267616A (en) * | 2020-02-28 | 2020-06-12 | 华域视觉科技(上海)有限公司 | Vehicle-mounted head-up display module and method and vehicle |
| JP2020095155A (en) * | 2018-12-12 | 2020-06-18 | 株式会社デンソー | Virtual image display system, display control device and display control program |
| CN111443490A (en) * | 2020-04-15 | 2020-07-24 | 诸暨市华鲟电子科技有限公司 | Virtual image display area adjusting method of AR HUD |
| WO2020158034A1 (en) * | 2019-02-01 | 2020-08-06 | 株式会社Ｊｖｃケンウッド | Display device and method for installing display device |
| CN111638786A (en) * | 2020-05-29 | 2020-09-08 | 电装天研究开发(天津)有限公司 | Display control method, device and equipment of vehicle-mounted rear projection display system and storage medium |
| CN112162688A (en) * | 2020-08-20 | 2021-01-01 | 江苏大学 | Vehicle-mounted virtual screen interactive information system based on gesture recognition |
| CN112241204A (en) * | 2020-12-17 | 2021-01-19 | 宁波均联智行科技有限公司 | Gesture interaction method and system of vehicle-mounted AR-HUD |
-
2021
- 2021-03-26 CN CN202110332139.6A patent/CN112947761B/en active Active
Patent Citations (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104145232A (en) * | 2012-01-04 | 2014-11-12 | 托比技术股份公司 | System for gaze interaction |
| CN103869970A (en) * | 2012-12-18 | 2014-06-18 | 现代自动车株式会社 | System and method for manipulating user interface by 2d camera |
| CN103885580A (en) * | 2012-12-20 | 2014-06-25 | 现代自动车株式会社 | Control system for using hand gesture for vehicle |
| CN105786361A (en) * | 2014-12-23 | 2016-07-20 | 北京中博安达知识产权服务有限公司 | 3D vehicle-mounted terminal man-machine interaction system |
| CN104875680A (en) * | 2015-06-03 | 2015-09-02 | 深圳市光晕网络科技有限公司 | HUD (head up display) device combining voice and video recognition |
| CN204821176U (en) * | 2015-07-17 | 2015-12-02 | 樊强 | But human -computer interaction's new line display |
| WO2017134876A1 (en) * | 2016-02-05 | 2017-08-10 | 日立マクセル株式会社 | Head-up display device and display control method therefor |
| CN106445104A (en) * | 2016-08-25 | 2017-02-22 | 蔚来汽车有限公司 | Vehicle HUD display system and method |
| CN106353886A (en) * | 2016-10-12 | 2017-01-25 | 惠州市华阳多媒体电子有限公司 | Intelligent head up display of automobile |
| JP2019032362A (en) * | 2017-08-04 | 2019-02-28 | アルパイン株式会社 | Head-up display device and navigation device |
| WO2019224922A1 (en) * | 2018-05-22 | 2019-11-28 | 三菱電機株式会社 | Head-up display control device, head-up display system, and head-up display control method |
| CN109445568A (en) * | 2018-08-30 | 2019-03-08 | 丝路视觉科技股份有限公司 | Projection objects control method, device and host |
| CN111038402A (en) * | 2018-10-11 | 2020-04-21 | 现代自动车株式会社 | Apparatus and method for controlling display of vehicle |
| CN109649275A (en) * | 2018-11-29 | 2019-04-19 | 福瑞泰克智能系统有限公司 | A kind of driving assistance system and method based on augmented reality |
| JP2020095155A (en) * | 2018-12-12 | 2020-06-18 | 株式会社デンソー | Virtual image display system, display control device and display control program |
| WO2020158034A1 (en) * | 2019-02-01 | 2020-08-06 | 株式会社Ｊｖｃケンウッド | Display device and method for installing display device |
| CN110471532A (en) * | 2019-08-19 | 2019-11-19 | 业成科技（成都）有限公司 | Integrate the car-mounted device of eyeball tracking Yu gesture manipulation |
| CN111158491A (en) * | 2019-12-31 | 2020-05-15 | 苏州莱孚斯特电子科技有限公司 | Gesture recognition man-machine interaction method applied to vehicle-mounted HUD |
| CN111267616A (en) * | 2020-02-28 | 2020-06-12 | 华域视觉科技(上海)有限公司 | Vehicle-mounted head-up display module and method and vehicle |
| CN111443490A (en) * | 2020-04-15 | 2020-07-24 | 诸暨市华鲟电子科技有限公司 | Virtual image display area adjusting method of AR HUD |
| CN111638786A (en) * | 2020-05-29 | 2020-09-08 | 电装天研究开发(天津)有限公司 | Display control method, device and equipment of vehicle-mounted rear projection display system and storage medium |
| CN112162688A (en) * | 2020-08-20 | 2021-01-01 | 江苏大学 | Vehicle-mounted virtual screen interactive information system based on gesture recognition |
| CN112241204A (en) * | 2020-12-17 | 2021-01-19 | 宁波均联智行科技有限公司 | Gesture interaction method and system of vehicle-mounted AR-HUD |
Non-Patent Citations (5)
| Title |
|---|
| KYEONGHOON PARK等: "AR-HUD system for tower crane on construction field", 2011 IEEE INTERNATIONAL SYMPOSIUM ON VR INNOVATION * |
| 景春晖;张晶;邓椿山;: "汽车手势交互设计研究", 包装工程, no. 08 * |
| 杨飞;林绍华;吴春建;: "抬头显示系统在汽车上的应用", 轻型汽车技术, no. 4 * |
| 陈璐玲;: "某车载抬头显示光学系统设计研究", 中国新技术新产品, no. 13 * |
| 鲁宇;张靖;冉舒文;邹耀;郑立兴;陈伟民;: "抬头显示系统成像参数测量技术现状分析", 激光杂志, no. 02 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112947761B (en) | 2023-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9878667B2 (en) | In-vehicle display apparatus and program product | |
| US20160124231A1 (en) | Head-up display device | |
| US11250816B2 (en) | Method, device and computer-readable storage medium with instructions for controlling a display of an augmented-reality head-up display device for a transportation vehicle | |
| US20210104212A1 (en) | Display control device, and nontransitory tangible computer-readable medium therefor | |
| JP2017215816A (en) | Information display device, information display system, information display method, and program | |
| KR20130089139A (en) | Augmented reality head-up display apparatus and method for vehicles | |
| US20190339535A1 (en) | Automatic eye box adjustment | |
| US10453176B2 (en) | Display apparatus to control display form of virtual object | |
| CN111252074B (en) | Multi-modal control method, device, computer-readable storage medium and vehicle | |
| US9826206B2 (en) | Display control device, projection device, and non-transitory storage medium | |
| CN112558305B (en) | Control method, device and medium of display picture and head-up display control system | |
| JP2018144554A (en) | Head-up display device for vehicle | |
| CN113064279B (en) | Virtual image position adjusting method, device and storage medium of AR-HUD system | |
| JPWO2020105685A1 (en) | Display controls, methods, and computer programs | |
| CN107472140B (en) | Eye-control vehicle-mounted HUD system | |
| US9495871B2 (en) | Display control device, display control method, non-transitory recording medium, and projection device | |
| JP2018151903A (en) | Virtual image display device and computer program | |
| CN113002461A (en) | Virtual image position adjusting method, device and storage medium of AR-HUD system | |
| US11276378B2 (en) | Vehicle operation system and computer readable non-transitory storage medium | |
| CN112947761B (en) | Virtual image position adjustment method, device and storage medium of AR-HUD system | |
| CN111923835A (en) | Vehicle-based rearview display method | |
| CN111435269A (en) | Display adjusting method, system, medium and terminal of vehicle head-up display device | |
| CN115018942A (en) | Method and apparatus for image display of vehicle | |
| KR20230034448A (en) | Vehicle and method for controlling thereof | |
| JP2016131009A (en) | Display control device, projection device, display control method, display control program, and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |