CN111107332A - HUD projection image display method and device - Google Patents
HUD projection image display method and device Download PDFInfo
- Publication number
- CN111107332A CN111107332A CN201911388451.6A CN201911388451A CN111107332A CN 111107332 A CN111107332 A CN 111107332A CN 201911388451 A CN201911388451 A CN 201911388451A CN 111107332 A CN111107332 A CN 111107332A
- Authority
- CN
- China
- Prior art keywords
- hud
- projection image
- image
- hud projection
- superposed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3188—Scale or resolution adjustment
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Controls And Circuits For Display Device (AREA)
- Instrument Panels (AREA)
Abstract
The invention discloses a HUD projection image display method, which comprises the following steps: setting a projection area of the HUD projection image on the target object; calculating state information of the object to be superposed based on a preset coordinate system; the object to be superposed is an object in a road condition where the target object is located, and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superposed, which is covered by the projection area, and a distance between the object to be superposed and the target object; in the running process of the target object, adjusting the size of the corresponding HUD projection image according to the state information of the object to be superposed; and projecting the HUD projection image after the size is adjusted to the projection area for displaying. The invention also discloses a HUD projection image display device, and by adopting the embodiment of the invention, the size of the HUD projection image can be adjusted in real time according to the change of road conditions, so that the visual experience of a user is improved.
Description
Technical Field
The invention relates to the technical field of image projection, in particular to a HUD projection image display method and device.
Background
With the continuous progress of technology, Head Up Di sp ay (HUD) is gradually applied to the fields of automobiles, aviation, and the like. The HUD of the existing mass production usually forms a virtual image by projection on a windshield, and the picture in the HUD is pasted on a virtual image surface for display. Before displaying the picture on the virtual image surface, all coordinate systems are integrated into a unified coordinate system, and then AR projection calculation is carried out, so that the picture is presented on the virtual image surface in a 3D mode. However, the 3D image generated by HUD calculation at present lacks the ability to adjust the size of the projected image in real time, and when the projection size of the image is too large or too small, the driver is likely to feel a foreign object, so that the visual experience is poor, and even the road condition misjudgment may be caused.
Disclosure of Invention
The embodiment of the invention aims to provide a HUD projection image display method and device, which can adjust the size of a HUD projection image in real time according to the change of road conditions and improve the visual experience of a user.
In order to achieve the above object, an embodiment of the present invention provides a HUD projection image display method, including:
setting a projection area of the HUD projection image on the target object;
calculating state information of the object to be superposed based on a preset coordinate system; the object to be superposed is an object in a road condition where the target object is located, and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superposed, which is covered by the projection area, and a distance between the object to be superposed and the target object;
in the running process of the target object, adjusting the size of the corresponding HUD projection image according to the state information of the object to be superposed;
and projecting the HUD projection image after the size is adjusted to the projection area for displaying.
Compared with the prior art, the HUD projection image display method disclosed by the embodiment of the invention comprises the steps of firstly, setting a projection area of the HUD projection image on a target object, wherein the projection area is used for superposing an AR (augmented reality) image projected by the HUD; then, calculating the state information of the object to be superposed based on a preset coordinate system, wherein the object to be superposed is a real object in the road condition; finally, in the target object operation process, adjust the size of the HUD projection image that corresponds according to the state information of waiting to superpose the object to the HUD projection image after will adjusting the size is projected and is shown to the projection area, superposes the HUD projection image and shows on the material object, strengthens the stack effect of AR image, can also adjust the size of HUD projection image in real time according to the change of road conditions, improves user's visual experience.
As an improvement of the above scheme, when the state information is a coverage range of an area formed by coordinate points of the object to be superimposed and covered by the projection area, the adjusting the size of the corresponding HUD projection image according to the state information of the object to be superimposed specifically includes:
and when the area formed by the coordinate points of the object to be superposed cannot be completely covered by the projection area, cutting the HUD projection image.
As an improvement of the above scheme, the cutting the HUD projection image specifically includes:
dividing the HUD projection image into a plurality of image points;
deleting image points beyond the projection area;
and connecting image points which do not exceed the projection area to form a cut HUD projection image.
As an improvement of the above scheme, when the state information is the distance between the object to be superimposed and the target object, the adjusting the size of the corresponding HUD projection image according to the state information of the object to be superimposed specifically includes:
when the distance between the object to be superposed and the target object is reduced, carrying out reduction operation on the HUD projection image;
and when the distance between the object to be superposed and the target object is increased, carrying out amplification operation on the HUD projection image.
As an improvement of the above scheme, the reducing operation performed on the HUD projection image specifically includes:
dividing the HUD projection image into a plurality of image points;
selecting an image point from the plurality of image points as a first target image point;
keeping the coordinates of the first target image point unchanged, and adjusting the coordinates of other image points according to a preset reduction scale;
and connecting the first target image point and the other image points after the coordinate adjustment to form a reduced HUD projection image.
As an improvement of the above scheme, the enlarging operation performed on the HUD projection image specifically includes:
dividing the HUD projection image into a plurality of image points;
selecting one image point from the plurality of image points as a second target image point;
keeping the coordinates of the second target image point unchanged, and adjusting the coordinates of other image points according to a preset amplification scale;
and connecting the second target image point and the other image points after the coordinate adjustment to form an enlarged HUD projection image.
As an improvement of the above scheme, the preset coordinate system is a human eye coordinate system; the target object is a vehicle.
To achieve the above object, an embodiment of the present invention further provides a HUD projection image display device including:
a projection region setting unit for setting a projection region of the HUD projection image on the target object;
the state information calculating unit is used for calculating the state information of the object to be superposed based on a preset coordinate system; the object to be superposed is an object in a road condition where the target object is located, and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superposed, which is covered by the projection area, and a distance between the object to be superposed and the target object;
the size adjusting unit is used for adjusting the size of the corresponding HUD projection image according to the state information of the object to be superposed in the running process of the target object;
and the projection unit is used for projecting the HUD projection image after the size adjustment to the projection area for display.
Compared with the prior art, the HUD projection image display device disclosed by the embodiment of the invention comprises a projection area setting unit, a display unit and a display unit, wherein the projection area setting unit is used for setting a projection area of the HUD projection image on a target object, and the projection area is used for superposing an AR (augmented reality) image projected by the HUD; then, the state information calculating unit calculates the state information of the object to be superposed based on a preset coordinate system, wherein the object to be superposed is a real object in the road condition; finally, in the target object operation process, the size of the HUD projection image that size adjustment unit corresponds according to the state information adjustment of waiting to superpose the object to the HUD projection image after projection unit will adjust the size is projected and is shown to the projection area, superposes the HUD projection image and shows on the material object, strengthens the stack effect of AR image, can also adjust the size of HUD projection image in real time according to the change of road conditions, improves user visual experience.
As an improvement of the above aspect, the size adjustment unit includes:
the cutting module is used for cutting the HUD projection image when an area formed by the coordinate points of the object to be superposed cannot be completely covered by the projection area;
the reduction module is used for carrying out reduction operation on the HUD projection image when the distance between the object to be superposed and the target object is reduced;
and the amplifying module is used for amplifying the HUD projection image when the distance between the object to be superposed and the target object is increased.
As an improvement of the above scheme, the preset coordinate system is a human eye coordinate system; the target object is a vehicle.
Drawings
Fig. 1 is a flowchart of a HUD projection image display method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a third coordinate system in a HUD projection image display method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a human eye coordinate system in a HUD projection image display method according to an embodiment of the present invention;
fig. 4 is a schematic diagram of positions of a projection area and an object to be superimposed in a HUD projection image display method according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating superimposition of HUD projection images and an object to be superimposed in the HUD projection image display method according to the embodiment of the present invention;
fig. 6 is a schematic diagram of the HUD projection image before being clipped in the HUD projection image display method according to the embodiment of the present invention;
fig. 7 is a schematic diagram of a HUD projection image after being clipped in the HUD projection image display method according to the embodiment of the present invention;
fig. 8 is a schematic diagram illustrating a reduction operation performed on a HUD projection image in a HUD projection image display method according to an embodiment of the present invention;
fig. 9 is a schematic diagram illustrating a enlarging operation of the HUD projection image in the HUD projection image display method according to the embodiment of the present invention;
fig. 10 is a block diagram showing the construction of a HUD projection image display apparatus according to an embodiment of the present invention;
fig. 11 is a block diagram showing a configuration of a size adjustment unit in the HUD projection image display apparatus according to the embodiment of the present invention;
fig. 12 is a block diagram showing the structure of another HUD projection image display apparatus according to the embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of a HUD projection image display method according to an embodiment of the present invention; the HUD projection image display method includes:
s1, setting a projection area of the HUD projection image on the target object;
s2, calculating the state information of the object to be superposed based on a preset coordinate system;
s3, in the running process of the target object, adjusting the size of the corresponding HUD projection image according to the state information of the object to be superposed;
and S4, projecting the HUD projection image after the size adjustment to the projection area for displaying.
It should be noted that the HUD projection image display method according to the embodiment of the present invention may be implemented by a controller installed in the HUD, or another control device connected to the HUD and configured to control the HUD projection image to be displayed. The target object in the embodiment of the invention is a vehicle, but in other embodiments of the invention, the target object may also be a vehicle such as an airplane or a ship.
Before the HUD projection image display method is executed, the coordinate systems involved in the target object need to be calibrated, so that the coordinate systems are all converted into a unified coordinate system, for example, the coordinate systems can be integrated into a human eye coordinate system, and then projection calculation is performed. The method has the advantages that the method can be kept consistent with a coordinate system of a projection area, the projection area is not fixed relative to a vehicle body, the environment outside the vehicle and human eyes are easy to move, and therefore the design can reduce the calculation workload and optimize the calculation efficiency.
Preferably, in the embodiment of the present invention, a first image capturing device and a second image capturing device are disposed in the target object, and the first image capturing device and the second image capturing device are respectively connected to the HUD. Preferably, the first camera device is a DMS camera, and the second camera device is an ADAS camera.
The first camera device is installed at the front end of the target object, faces a driver, and is used for collecting driver information, such as collecting human eye position information of the driver. The first camera device is also used for establishing a first coordinate system of the first camera device. Taking a vehicle as an example, the first coordinate system takes the first camera device as an origin, the rear of the vehicle is a positive X-axis direction, the right of the vehicle is a positive Y-axis direction, and the right above is a positive Z-axis direction. And the first camera device is used for acquiring the position information of human eyes in the first coordinate system and sending the position information to the HUD.
The second camera device is installed at the front end of the target object, faces the road surface, and is used for acquiring road condition information and sending the road condition information to the HUD. The first camera device is also used for establishing a second coordinate system of the first camera device, the second coordinate system takes the second camera device as an origin, the advancing direction of the vehicle is the positive direction of an X axis, the positive right direction of the vehicle is the positive direction of a Y axis, and the positive lower side of the vehicle is the positive direction of a Z axis. It can be understood that the traffic information may include position information, color information, and the like of the object information to be superimposed, then, the second camera device acquires the position information of the object information to be superimposed in the second coordinate system and sends the position information to the HUD, the position information is coordinate information of an area formed by coordinate points of the object to be superimposed, and the second camera device may also send the color information of the object information to be superimposed to the HUD.
Exemplarily, the object to be superposed is a road, another vehicle, a building, a tree, etc., the second camera device can acquire an actual object, and the HUD projection image is an AR image. The object to be superimposed may include a plurality of HUD projection images, and a plurality of HUD projection images may be correspondingly displayed in the projection region.
The HUD is installed in the position, facing the windshield, of the target object and used for projecting the HUD projection image onto the windshield of the target object so as to display the HUD in a superposition mode with an external real object. The HUD has established the third coordinate system, the third coordinate system with the regional upper left corner of projection in the windshield is the origin, refer to figure 2, and figure 2 is when driving position department people's eye E (X, Y) faces the projection region, the positive direction of X and Y axle in the third coordinate system.
Preferably, the HUD converts the coordinate information in all coordinate systems into the position information in the preset coordinate system. For example, when the preset coordinate system is a human eye coordinate system, after the human eye position information sent by the first camera device is obtained, the human eye coordinate system may be established based on the human eye position information, as shown in fig. 3, the human eye coordinate system uses the eye middle point as a coordinate origin O, the vehicle forward direction is a positive X-axis direction, the vehicle right-hand direction is a positive Y-axis direction, and the right-lower direction is a positive Z-axis direction. HUD can all convert the coordinate information of other received coordinate systems into coordinate information under the people's eye coordinate system, carry out simple coordinate translation can, reduce calculation work load, optimize calculation efficiency.
Exemplarily, the first camera device has a position (Xec, Yec, Zec) in the human eye coordinate system; the position of the second camera in the human eye coordinate system is (Xec + Xad, Yec + Yad, Zec + Zad); the set of image points of said HUD projection image is defined as (Xpn, Ypn, Zpn), Xpn ═ Xrpn + Xec + Xad, Ypn ═ Yrpn + Yec + Yad, Zpn ═ Zrpn + Zec + Zad; the HUD projection image has central coordinates of (Xec + Xar, Yec + Yar, Zec + Zar); the distance between the road surface and human eyes is Zec + Zrd; after coordinate conversion, the HUD projection image coordinates are as follows: upper left (Xec + Xar, Yec + Yar-W/2, Zec + Zar-H/2), upper right (Xec + Xar, Yec + Yar + W/2, Zec + Zar-H/2), lower left (Xec + Xar, Yec + Yar-W/2, Zec + Zar + H/2), lower right (Xec + Xar, Yec + Yar + W/2, Zec + Zar + H/2).
Specifically, in step S1, the projection area of the HUD projection image on the target object is set. The projection area may be provided at the windshield directly in front of the driver, and is imaged at a position at a distance from the human eyes by refraction. The projection area may be preset before starting the HUD and needs to be determined according to the parameters provided by the HUD. A parameter that can be referenced is imaging at 45 ° at 7.5 meters from the human eye. The driver can adjust the position of the projection area by adjusting the mounting position of the HUD, and the projection area remains unchanged if the mounting position of the HUD is not moved.
It is understood that the HUD projection image corresponding to the object to be superimposed is the virtual object with the superimposed object, and may have a similar or the same appearance feature as the object to be superimposed. For example, when the object to be superimposed is a curved road, the HUD projection image corresponding to the object to be superimposed is a curved curve, and the direction and angle of the curve are the same as those of the road; for example, when the object to be superimposed is a roadside building, the HUD projection image corresponding thereto is a building outline, and is obtained by reducing the building outline in a certain proportion according to a real object. Therefore, the current road condition information can be correspondingly displayed on the windshield.
Specifically, in step S2, the state information of the object to be superimposed is calculated based on the preset coordinate system. Preferably, the object to be superimposed is an object in the road condition where the target object is located (i.e., an object in the road condition information acquired by the second camera device), and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superimposed, the coverage range being covered by the projection area, and a distance between the object to be superimposed and the target object.
For example, after the position information (the coordinate information in the second coordinate system) of the object to be superimposed, which is sent by the second imaging device, is obtained, the coordinate information of the region formed by the coordinate points of the object to be superimposed and the coordinate information of the projection region (the coordinate information in the third coordinate system) are converted into the coordinate information in the human eye coordinate system, so that comparison can be performed in one coordinate system, and the coverage range of the region formed by the coordinate points of the object to be superimposed, which is covered by the projection region in the human eye viewing angle, is obtained.
Specifically, in step S3, when the HUD is turned on during the operation of the target object, the size of the HUD projection image is adjusted according to the state information of the object to be superimposed. It will be appreciated that when the target object remains stationary and the HUD is turned on, such as a red light in a vehicle, the HUD projection image remains in the size as it was displayed at the previous time. When the HUD is turned off, such as when the vehicle is powered down, the HUD projection images cease to be displayed.
In a preferred embodiment, when the state information is a coverage range of a region formed by coordinate points of the object to be superimposed and covered by the projection region, the adjusting the size of the corresponding HUD projection image according to the state information of the object to be superimposed specifically includes: and when the area formed by the coordinate points of the object to be superposed cannot be completely covered by the projection area, cutting the HUD projection image.
Illustratively, referring to fig. 4, the coordinate system includes a driver 100, a projection area 200, and an area 300 (taking a part of a road as an example) composed of coordinate points of the objects to be superimposed, where point 0 in the figure is an origin of the human eye coordinate system. Wherein, regarding the selected area 300, the area 300 may be composed by selecting coordinate points of several boundaries of the objects to be superimposed. The dotted lines L1 and L2 in the figure indicate the upper and lower ranges that the projection area 200 can cover. In the driving process of the vehicle, the original point under the coordinates of the human eyes is used as a visual starting point, the coordinates of the area 300 consisting of the coordinate points of the objects to be superposed and the projection area 200 are converted into the coordinates under the coordinates of the human eyes, and the coverage area of the area 300 covered by the projection area can be calculated. Referring now to fig. 5, the arrowed curve in the figure is the HUD projection image, which is used to indicate the driving direction of the driver. During the driving process of the vehicle, the area 300 formed by the current part of the road gradually leaves the coverage area of the projection area 200 (as shown in the right diagram in fig. 5) as the vehicle approaches, and at this time, the HUD projection image needs to be cut correspondingly, that is, the image beyond the coverage area of the projection area 200 is cut. It should be noted that, for convenience, the HUD projection image beyond the projection area 200 is not cropped in fig. 5, but in the implementation, the portion beyond the projection area 200 is not displayed.
Preferably, the cropping the HUD projection image specifically includes:
s311, dividing the HUD projection image into a plurality of image points. As shown in fig. 6, the "×" shaped graph in the figure is the HUD projection image, and turning points and/or connection points in the contour of the HUD projection image are selected as image points, such as P0 to P6 in the figure, which are totally divided into 7 image points.
And S312, deleting the image points beyond the projection area. For example, two image points P5 and P6 are not in the projection region, so that two image points P5 and P6 need to be deleted. It is understood that the image points to be retained are image points that can form a graph (triangle, rectangle, etc.), for example, the triangle in the graph needs to be formed by using three image points P0, P1, and P2, and if P2 is not projected on the projection area, the triangle will be clipped and not displayed. The graphics are composed of P0, P1, P2, P3, P4, P5 and P6, but only P0, P1, P2, P3 and P4 are needed to form a complete image, so that the residual graphics can be displayed after P5 and P6 are cut.
And S313, connecting the image points which do not exceed the projection area to form a cut HUD projection image. As shown in fig. 7, the clipped HUD projection image is obtained by connecting the remaining image points P0, P1, P2, P3, and P4.
In another preferred embodiment, when the state information is the distance between the object to be superimposed and the target object, the adjusting the size of the corresponding HUD projection image according to the state information of the object to be superimposed specifically includes: when the distance between the object to be superposed and the target object is reduced, carrying out reduction operation on the HUD projection image; and when the distance between the object to be superposed and the target object is increased, carrying out amplification operation on the HUD projection image.
For example, during the driving of the vehicle, the object to be superimposed visually appears to be gradually smaller as the vehicle moves away (the vehicle moves backwards), and at the moment, the distance between the object to be superimposed and the vehicle is larger, and the corresponding HUD projection image is correspondingly reduced; the object to be superimposed gradually becomes larger as the vehicle approaches (the vehicle advances), the distance between the object to be superimposed and the vehicle becomes smaller, and the corresponding HUD projection image is correspondingly enlarged. And zooming the HUD projection image according to the distance between the real object in the road condition information and the target object, so as to enhance the visual experience of the user.
Preferably, the reducing operation performed on the HUD projection image specifically includes:
s321, dividing the HUD projection image into a plurality of image points. As shown in fig. 8, the "×" shaped graph in the figure is the HUD projection image, and turning points and/or connection points in the contour of the HUD projection image are selected as image points, for example, P0 to P6 in the figure are divided into 7 image points.
S322, selecting one image point from the plurality of image points as a first target image point. Such as P0, is selected as the first target image point.
And S323, keeping the coordinate of the first target image point unchanged, and adjusting the coordinates of other image points according to a preset reduction scale. The coordinates of P0 are adjusted according to the preset reduction ratio while keeping the coordinates of P0 unchanged, and the coordinates of P1, P2, P3, P4, P5, and P6 are adjusted, and the HUD projection image after the coordinates are adjusted is as shown in the right diagram of fig. 8.
And S324, connecting the first target image point and the other image points after the coordinate adjustment to form a reduced HUD projection image.
Preferably, the enlarging operation performed on the HUD projection image specifically includes:
s331, dividing the HUD projection image into a plurality of image points. As shown in fig. 9, the "×" shaped graph in the figure is the HUD projection image, and turning points and/or connection points in the contour of the HUD projection image are selected as image points, for example, P0 to P6 in the figure are divided into 7 image points.
S332, selecting one image point from the plurality of image points as a second target image point. Such as P0, is selected as the second target image point.
S333, keeping the coordinates of the second target image point unchanged, and adjusting the coordinates of other image points according to a preset amplification scale. The coordinates of P0 are kept unchanged, the coordinates of P1, P2, P3, P4, P5 and P6 are adjusted according to the preset magnification ratio, and the HUD projection image after the coordinates are adjusted is as shown in the right diagram in fig. 9.
And S334, connecting the second target image point and the other image points after the coordinates are adjusted to form an enlarged HUD projection image.
It is to be understood that, when the HUD projection image is cropped, the reduction or enlargement operation may be performed on the HUD projection image at the same time. For example, when the vehicle moves forward, the object to be superimposed gradually approaches the vehicle, and then the HUD projection image needs to be enlarged, and meanwhile, as the vehicle gradually approaches, the object to be superimposed may exceed the coverage area covered by the projection area, and then the HUD projection image needs to be clipped at the same time.
Specifically, in step S4, the HUD projection image after the size adjustment is projected onto the projection area and displayed. And displaying the HUD projection image in the projection area after the size of the HUD projection image is adjusted, so that the visual error of a user caused by incomplete imaging can be avoided.
Further, after setting the projection area of the HUD projection image on the target object, the HUD projection image display method further includes:
s21, calculating the orientation information of the object to be superposed based on a preset coordinate system; the orientation information represents which direction of the object to be superposed is in the target object by taking the target object as a reference object, and the orientation information of the object to be superposed can be obtained according to the position information of the object to be superposed, which is sent by the second camera device;
s22, in the running process of the target object, adjusting the display position of the corresponding HUD projection image according to the azimuth information of the object to be superposed;
and S23, projecting the HUD projection image with the display position adjusted to the projection area for display.
For example, when the vehicle is moving forward/backward, the orientation information of the object to be superimposed may change, and the display position of the HUD projection image in the projection area may be correspondingly adjusted. For example, when the object to be superimposed moves to the right, the HUD projection image also moves to the right in the projection region; when the object to be superimposed moves to the left, the HUD projection image also moves to the left in the projection area, and the HUD projection image can perform coordinate translation while performing zoom-in/zoom-out operations.
Furthermore, when adjusting HUD projection image size and/or display position, can be right the HUD projection image carries out the color adjustment, can be according to this moment wait to superpose the color information of object and correspond the adjustment, for example when current building is red, can correspond the HUD projection image that shows red, can strengthen the stack effect of HUD projection image and material object. Or, adjust the colour of HUD projecting image according to the current time, for example in daytime, because of the light is stronger, need strengthen the display colour of HUD projecting image this moment, can adopt darker colour to project the HUD projecting image, also can avoid using black to project the HUD projecting image night.
Compared with the prior art, the HUD projection image display method disclosed by the embodiment of the invention comprises the steps of firstly, setting a projection area of the HUD projection image on a target object, wherein the projection area is used for superposing an AR (augmented reality) image projected by the HUD; then, calculating the state information of the object to be superposed based on a preset coordinate system, wherein the object to be superposed is a real object in the road condition; finally, in the target object operation process, adjust the size of the HUD projection image that corresponds according to the state information of waiting to superpose the object to the HUD projection image after will adjusting the size is projected and is shown to the projection area, superposes the HUD projection image and shows on the material object, strengthens the stack effect of AR image, can also adjust the size of HUD projection image in real time according to the change of road conditions, improves user's visual experience.
Referring to fig. 10, fig. 10 is a block diagram showing a configuration of a HUD projection image display apparatus according to an embodiment of the present invention; the HUD projection image display device includes:
a projection region setting unit 10 for setting a projection region of the HUD projection image on the target object;
the state information calculating unit 20, is used for calculating the state information of the object to be superposed based on the preset coordinate system;
the size adjusting unit 30 is configured to adjust the size of the corresponding HUD projection image according to the state information of the object to be superimposed during the operation of the target object;
and the projection unit 40 is used for projecting the HUD projection image after the size adjustment to the projection area for displaying.
It should be noted that the HUD projection image display device according to the embodiment of the present invention may be a controller installed in the HUD, or another control device connected to the HUD and configured to control the HUD projection image to be displayed. The target object in the embodiment of the invention is a vehicle, but in other embodiments of the invention, the target object may also be a vehicle such as an airplane or a ship.
Before the HUD projection image is projected, the coordinate systems involved in the target object need to be calibrated to convert the coordinate systems into a unified coordinate system, for example, the coordinate systems can be integrated into the coordinate system of the human eye, and then projection calculation is performed. The method has the advantages that the method can be kept consistent with a coordinate system of a projection area, the projection area is not fixed relative to a vehicle body, the environment outside the vehicle and human eyes are easy to move, and therefore the design can reduce the calculation workload and optimize the calculation efficiency.
Preferably, in the embodiment of the present invention, a first image capturing device and a second image capturing device are disposed in the target object, and the first image capturing device and the second image capturing device are respectively connected to the HUD. Preferably, the first camera device is a DMS camera, and the second camera device is an ADAS camera.
The first camera device is installed at the front end of the target object, faces a driver, and is used for collecting driver information, such as collecting human eye position information of the driver. The first camera device is also used for establishing a first coordinate system of the first camera device. Taking a vehicle as an example, the first coordinate system takes the first camera device as an origin, the rear of the vehicle is a positive X-axis direction, the right of the vehicle is a positive Y-axis direction, and the right above is a positive Z-axis direction. And the first camera device is used for acquiring the position information of human eyes in the first coordinate system and sending the position information to the HUD projection image display device.
The second camera device is installed at the front end of the target object, faces the road surface, and is used for acquiring road condition information and sending the road condition information to the HUD projection image display device. The first camera device is also used for establishing a second coordinate system of the first camera device, the second coordinate system takes the second camera device as an origin, the advancing direction of the vehicle is the positive direction of an X axis, the positive right direction of the vehicle is the positive direction of a Y axis, and the positive lower side of the vehicle is the positive direction of a Z axis. It can be understood that the traffic information may include position information, color information, and the like of the object information to be superimposed, and then, the second camera device acquires the position information of the object information to be superimposed in the second coordinate system and sends the position information to the HUD projection image display device, and the second camera device may also send the color information of the object information to be superimposed to the HUD projection image display device.
Exemplarily, the object to be superposed is a road, another vehicle, a building, a tree, etc., the second camera device can acquire an actual object, and the HUD projection image is an AR image. The object to be superimposed may include a plurality of HUD projection images, and a plurality of HUD projection images may be correspondingly displayed in the projection region.
The HUD is installed in the position, facing the windshield, of the target object and used for projecting the HUD projection image onto the windshield of the target object so as to display the HUD in a superposition mode with an external real object. The HUD has established the third coordinate system, the third coordinate system with the regional upper left corner of projection in the windshield is the origin, refer to figure 2, and figure 2 is when driving position department people's eye E (X, Y) faces the projection region, the positive direction of X and Y axle in the third coordinate system.
Preferably, the HUD projection image display device converts coordinate information in all coordinate systems into position information in a preset coordinate system. For example, when the preset coordinate system is a human eye coordinate system, after the human eye position information sent by the first camera device is obtained, the human eye coordinate system may be established based on the human eye position information, as shown in fig. 3, the human eye coordinate system uses the eye middle point as a coordinate origin O, the vehicle forward direction is a positive X-axis direction, the vehicle right-hand direction is a positive Y-axis direction, and the right-lower direction is a positive Z-axis direction. HUD projection image display device can all convert the coordinate information of other received coordinate systems into coordinate information under the people's eye coordinate system, carry out simple coordinate translation can, reduce the calculation work load, optimize calculation efficiency.
Specifically, the projection region setting unit 10 sets a projection region of the HUD projection image on the target object. The projection area may be provided at the windshield directly in front of the driver, and is imaged at a position at a distance from the human eyes by refraction. The projection area may be preset before starting the HUD and needs to be determined according to the parameters provided by the HUD. A parameter that can be referenced is imaging at 45 ° at 7.5 meters from the human eye. The driver can adjust the position of the projection area by adjusting the mounting position of the HUD, and the projection area remains unchanged if the mounting position of the HUD is not moved.
It is understood that the HUD projection image corresponding to the object to be superimposed is the virtual object with the superimposed object, and may have a similar or the same appearance feature as the object to be superimposed. For example, when the object to be superimposed is a curved road, the HUD projection image corresponding to the object to be superimposed is a curved curve, and the direction and angle of the curve are the same as those of the road; for example, when the object to be superimposed is a roadside building, the HUD projection image corresponding thereto is a building outline, and is obtained by reducing the building outline in a certain proportion according to a real object. Therefore, the current road condition information can be correspondingly displayed on the windshield.
Specifically, the state information calculation unit 20 calculates the state information of the object to be superimposed based on a preset coordinate system. Preferably, the object to be superimposed is an object in the road condition where the target object is located (i.e., an object in the road condition information acquired by the second camera device), and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superimposed, the coverage range being covered by the projection area, and a distance between the object to be superimposed and the target object.
For example, after the position information (coordinate information in a second coordinate system) of the object to be superimposed sent by the second imaging device is acquired, the state information calculation unit 20 converts the position information of the object to be superimposed and the coordinate information (coordinate information in a third coordinate system) of the projection area into the coordinate information in the human eye coordinate system, so that comparison can be performed in one coordinate system to obtain a coverage area of an area formed by coordinate points of the object to be superimposed, the coverage area being covered by the projection area.
Specifically, the size adjusting unit 30 adjusts the size of the corresponding HUD projection image according to the state information of the object to be superimposed when the target object is in the operation process and the HUD is turned on. It will be appreciated that when the target object remains stationary and the HUD is turned on, such as a red light in a vehicle, the HUD projection image remains in the size as it was displayed at the previous time. When the HUD is turned off, such as when the vehicle is powered down, the HUD projection images cease to be displayed.
Referring to fig. 11, fig. 11 is a block diagram showing a configuration of a resizing unit 30 in a HUD projection image display apparatus according to an embodiment of the present invention; the size adjusting unit 30 includes:
and the cutting module 31 is configured to cut the HUD projection image when the area formed by the coordinate points of the object to be superimposed is not completely covered by the projection area.
And a reduction module 32, configured to perform a reduction operation on the HUD projection image when the distance between the object to be superimposed and the target object is reduced.
And the amplifying module 33 is configured to perform an amplifying operation on the HUD projection image when the distance between the object to be superimposed and the target object is increased.
Preferably, the cutting module 31 is specifically configured to: dividing the HUD projection image into a plurality of image points; deleting image points beyond the projection area; connecting image points not exceeding the projection area to form a cropped HUD projection image
Preferably, the reduction module 32 is specifically configured to: dividing the HUD projection image into a plurality of image points; selecting an image point from the plurality of image points as a first target image point; keeping the coordinates of the first target image point unchanged, and adjusting the coordinates of other image points according to a preset reduction scale; and connecting the first target image point and the other image points after the coordinate adjustment to form a reduced HUD projection image.
Preferably, the amplifying module 33 is specifically configured to: dividing the HUD projection image into a plurality of image points; an image point is selected from the plurality of image points as a second target image point. Such as selecting P0 as the second target image point; keeping the coordinates of the second target image point unchanged, and adjusting the coordinates of other image points according to a preset amplification scale; and connecting the second target image point and the other image points after the coordinate adjustment to form an enlarged HUD projection image.
It is to be understood that, when the cropping module 31 crops the HUD projection image, the reduction module 32 may perform a reduction operation on the HUD projection image at the same time, or the enlargement module 33 may perform an enlargement operation on the HUD projection image at the same time. For example, when the vehicle moves forward, the object to be superimposed gradually approaches the vehicle, the enlarging module 33 needs to enlarge the HUD projection image, and as the vehicle gradually approaches, the area formed by the coordinate points of the object to be superimposed may exceed the coverage area covered by the projection area, and the cropping module 31 needs to crop the HUD projection image at the same time.
Specifically, the projection unit 40 projects the HUD projection image after the size adjustment to the projection area for display. And displaying the HUD projection image in the projection area after the size of the HUD projection image is adjusted, so that the visual error of a user caused by incomplete imaging can be avoided.
Further, referring to fig. 12, the HUD projection image display device further includes:
an orientation information calculation unit 50 for calculating orientation information of the object to be superimposed based on a preset coordinate system; the orientation information represents which direction of the object to be superposed is in the target object by taking the target object as a reference object, and the orientation information of the object to be superposed can be obtained according to the position information of the object to be superposed, which is sent by the second camera device;
a display position adjusting unit 60, configured to adjust a display position of the corresponding HUD projection image according to the orientation information of the object to be superimposed during operation of the target object;
the projection unit 40 is further configured to project the HUD projection image with the display position adjusted to the projection area for display.
For example, when the vehicle is moving forward/backward, the orientation information of the object to be superimposed may change, and the display position of the HUD projection image in the projection area may be correspondingly adjusted. For example, when the object to be superimposed moves to the right, the HUD projection image also moves to the right in the projection region; when the object to be superimposed moves to the left, the HUD projection image also moves to the left in the projection area, and the HUD projection image can perform coordinate translation while performing zoom-in/zoom-out operations.
Further, the HUD projection image display device further includes:
a color adjustment unit 70 for performing color adjustment on the HUD projection image.
When adjusting HUD projection image size and/or display position, color adjustment unit 70 can correspond the adjustment according to the color information of waiting to superpose the object, for example when current building is red, can correspond and show red HUD projection image, can strengthen the stack effect of HUD projection image and real object. Or, adjust the colour of HUD projecting image according to the current time, for example in daytime, because of the light is stronger, need strengthen the display colour of HUD projecting image this moment, can adopt darker colour to project the HUD projecting image, also can avoid using black to project the HUD projecting image night.
Compared with the prior art, the HUD projection image display device disclosed in the embodiment of the present invention first sets the projection area of the HUD projection image on the target object, where the projection area is used to superimpose the AR image projected by the HUD; then, the state information calculating unit 20 calculates the state information of the object to be superimposed based on the preset coordinate system, wherein the object to be superimposed is a real object in the road condition; finally, at target object operation in-process, size adjustment unit 30 is according to the size of the HUD projecting image that the state information adjustment of waiting to superpose the object corresponds to the HUD projecting image projection that projection unit 40 will adjust after the size shows to the projection area, superposes the HUD projecting image and shows in kind on the material object, strengthens the stack effect of AR image, can also adjust the size of HUD projecting image in real time according to the change of road conditions, improves user visual experience.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (10)
1. A HUD projection image display method characterized by comprising:
setting a projection area of the HUD projection image on the target object;
calculating state information of the object to be superposed based on a preset coordinate system; the object to be superposed is an object in a road condition where the target object is located, and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superposed, which is covered by the projection area, and a distance between the object to be superposed and the target object;
in the running process of the target object, adjusting the size of the corresponding HUD projection image according to the state information of the object to be superposed;
and projecting the HUD projection image after the size is adjusted to the projection area for displaying.
2. The HUD projection image display method according to claim 1, wherein when the status information is a coverage area of an area formed by coordinate points of the object to be superimposed, which is covered by the projection area, the adjusting the size of the corresponding HUD projection image according to the status information of the object to be superimposed specifically includes:
and when the area formed by the coordinate points of the object to be superposed cannot be completely covered by the projection area, cutting the HUD projection image.
3. The HUD projection image display method according to claim 2, wherein the cropping the HUD projection image specifically includes:
dividing the HUD projection image into a plurality of image points;
deleting image points beyond the projection area;
and connecting image points which do not exceed the projection area to form a cut HUD projection image.
4. The HUD projection image display method according to claim 1, wherein when the status information is a distance between the object to be superimposed and the target object, the adjusting the size of the corresponding HUD projection image according to the status information of the object to be superimposed specifically includes:
when the distance between the object to be superposed and the target object is reduced, carrying out reduction operation on the HUD projection image;
and when the distance between the object to be superposed and the target object is increased, carrying out amplification operation on the HUD projection image.
5. The HUD projection image display method according to claim 4, wherein the reducing operation performed on the HUD projection image specifically includes:
dividing the HUD projection image into a plurality of image points;
selecting an image point from the plurality of image points as a first target image point;
keeping the coordinates of the first target image point unchanged, and adjusting the coordinates of other image points according to a preset reduction scale;
and connecting the first target image point and the other image points after the coordinate adjustment to form a reduced HUD projection image.
6. The HUD projection image display method according to claim 4, wherein the enlarging operation performed on the HUD projection image specifically includes:
dividing the HUD projection image into a plurality of image points;
selecting one image point from the plurality of image points as a second target image point;
keeping the coordinates of the second target image point unchanged, and adjusting the coordinates of other image points according to a preset amplification scale;
and connecting the second target image point and the other image points after the coordinate adjustment to form an enlarged HUD projection image.
7. The HUD projection image display method according to claim 1, wherein the preset coordinate system is a human eye coordinate system; the target object is a vehicle.
8. A HUD projection image display device characterized by comprising:
a projection region setting unit for setting a projection region of the HUD projection image on the target object;
the state information calculating unit is used for calculating the state information of the object to be superposed based on a preset coordinate system; the object to be superposed is an object in a road condition where the target object is located, and the state information includes at least one of a coverage range of an area formed by coordinate points of the object to be superposed, which is covered by the projection area, and a distance between the object to be superposed and the target object;
the size adjusting unit is used for adjusting the size of the corresponding HUD projection image according to the state information of the object to be superposed in the running process of the target object;
and the projection unit is used for projecting the HUD projection image after the size adjustment to the projection area for display.
9. The HUD projection image display device according to claim 8, wherein the size adjustment unit includes:
the cutting module is used for cutting the HUD projection image when an area formed by the coordinate points of the object to be superposed cannot be completely covered by the projection area;
the reduction module is used for carrying out reduction operation on the HUD projection image when the distance between the object to be superposed and the target object is reduced;
and the amplifying module is used for amplifying the HUD projection image when the distance between the object to be superposed and the target object is increased.
10. A HUD projection image display apparatus according to claim 8, wherein the preset coordinate system is a human eye coordinate system; the target object is a vehicle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911388451.6A CN111107332A (en) | 2019-12-30 | 2019-12-30 | HUD projection image display method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911388451.6A CN111107332A (en) | 2019-12-30 | 2019-12-30 | HUD projection image display method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111107332A true CN111107332A (en) | 2020-05-05 |
Family
ID=70423722
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911388451.6A Pending CN111107332A (en) | 2019-12-30 | 2019-12-30 | HUD projection image display method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111107332A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022094739A1 (en) * | 2020-11-03 | 2022-05-12 | 谢建军 | Projection system, method, and apparatus, and computer device |
| WO2022241638A1 (en) * | 2021-05-18 | 2022-11-24 | 华为技术有限公司 | Projection method and apparatus, and vehicle and ar-hud |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101248473A (en) * | 2005-09-29 | 2008-08-20 | 松下电器产业株式会社 | Navigation device, navigation method, and vehicle |
| CN106226910A (en) * | 2016-09-08 | 2016-12-14 | 邹文韬 | HUD system and image regulating method thereof |
| CN108700744A (en) * | 2016-02-10 | 2018-10-23 | 株式会社理光 | Information display device |
| CN109427199A (en) * | 2017-08-24 | 2019-03-05 | 北京三星通信技术研究有限公司 | For assisting the method and device of the augmented reality driven |
-
2019
- 2019-12-30 CN CN201911388451.6A patent/CN111107332A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101248473A (en) * | 2005-09-29 | 2008-08-20 | 松下电器产业株式会社 | Navigation device, navigation method, and vehicle |
| CN108700744A (en) * | 2016-02-10 | 2018-10-23 | 株式会社理光 | Information display device |
| CN106226910A (en) * | 2016-09-08 | 2016-12-14 | 邹文韬 | HUD system and image regulating method thereof |
| CN109427199A (en) * | 2017-08-24 | 2019-03-05 | 北京三星通信技术研究有限公司 | For assisting the method and device of the augmented reality driven |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022094739A1 (en) * | 2020-11-03 | 2022-05-12 | 谢建军 | Projection system, method, and apparatus, and computer device |
| WO2022241638A1 (en) * | 2021-05-18 | 2022-11-24 | 华为技术有限公司 | Projection method and apparatus, and vehicle and ar-hud |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7010221B2 (en) | Image generator, image generation method, and program | |
| WO2012096058A1 (en) | Image generating device | |
| US8179435B2 (en) | Vehicle surroundings image providing system and method | |
| JP5397373B2 (en) | VEHICLE IMAGE PROCESSING DEVICE AND VEHICLE IMAGE PROCESSING METHOD | |
| JP5874920B2 (en) | Monitoring device for vehicle surroundings | |
| JP4248570B2 (en) | Image processing apparatus and visibility support apparatus and method | |
| CN106226910A (en) | HUD system and image regulating method thereof | |
| EP2476587B1 (en) | Vehicle surrounding monitor apparatus | |
| JP2004021307A (en) | Driving support device | |
| WO2000064175A1 (en) | Image processing device and monitoring system | |
| WO2021197190A1 (en) | Information display method, system and apparatus based on augmented reality, and projection device | |
| EP3326145B1 (en) | Panel transform | |
| EP3326146B1 (en) | Rear cross traffic - quick looks | |
| CN111107332A (en) | HUD projection image display method and device | |
| WO2019034916A1 (en) | System and method for presentation and control of virtual camera image for a vehicle | |
| JP5861871B2 (en) | Overhead image presentation device | |
| WO2021240872A1 (en) | Display control device, vehicle and display control method | |
| JP4706896B2 (en) | Wide-angle image correction method and vehicle periphery monitoring system | |
| JP4696825B2 (en) | Blind spot image display device for vehicles | |
| WO2022044374A1 (en) | Display control device, vehicle, and display control method | |
| US20240208415A1 (en) | Display control device and display control method | |
| WO2021240873A1 (en) | Display control device, vehicle, and display control method | |
| US20240214510A1 (en) | Image processing device and image processing method | |
| US20220161723A1 (en) | Parking assistance system | |
| JP2024090467A (en) | Display control device and display control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200505 |