CN109993713B - Image distortion correction method and device for vehicle-mounted head-up display system - Google Patents
Image distortion correction method and device for vehicle-mounted head-up display system Download PDFInfo
- Publication number
- CN109993713B CN109993713B CN201910270514.1A CN201910270514A CN109993713B CN 109993713 B CN109993713 B CN 109993713B CN 201910270514 A CN201910270514 A CN 201910270514A CN 109993713 B CN109993713 B CN 109993713B
- Authority
- CN
- China
- Prior art keywords
- image
- display system
- input image
- mounted head
- target vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012937 correction Methods 0.000 title claims abstract description 96
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 52
- 238000012545 processing Methods 0.000 claims description 67
- 230000006870 function Effects 0.000 claims description 32
- 238000013135 deep learning Methods 0.000 claims description 27
- 239000011159 matrix material Substances 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 3
- 230000003287 optical effect Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3185—Geometric adjustment, e.g. keystone or convergence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3188—Scale or resolution adjustment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Geometry (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Controls And Circuits For Display Device (AREA)
- Image Processing (AREA)
Abstract
The application provides a vehicle-mounted head-up display system image distortion correction method and device, wherein the method comprises the following steps: acquiring an image to be projected; correcting the image to be projected by using a preset correction model to obtain a corrected image to be projected; and inputting the corrected image to be projected into a target vehicle-mounted head-up display system for projection display, wherein a preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and the target vehicle-mounted head-up display system in a training mode. The method comprises the steps of firstly utilizing a correction model to correct an image before the image is subjected to projection display, then inputting the image into a vehicle-mounted head-up display system for projection display after the image is corrected, and utilizing the correction model to perform distortion correction.
Description
Technical Field
The application relates to the technical field of display, in particular to a method and a device for correcting image distortion of a vehicle-mounted head-up display system.
Background
With the development of technology, head-up display devices are increasingly used in vehicles, and a trend is gradually developed. A head-up display device on a vehicle is where an image is projected in front of the vehicle through a front windshield of the vehicle. Because the windshield is an irregular curved surface, the image projected outside the vehicle through the windshield can generate distortion and phase difference, so that the display quality of the image is poor.
Currently, correction is performed by multiple optical methods, such as adding a non-continuous curved mirror or other optical elements before the windshield to correct the reduced imaging effect caused by the windshield. However, this correction method requires different optical elements for different windshields, which not only makes the design difficult, but also increases the cost of the vehicle.
Disclosure of Invention
The application provides an image distortion correction method and device for a vehicle-mounted head-up display system, which are used for solving the problems of high design difficulty and high cost of a method for performing distortion correction by adding an optical element in the related art.
An embodiment of the application provides a method for correcting image distortion of a vehicle-mounted head-up display system, which comprises the following steps:
acquiring an image to be projected;
correcting the image to be projected by using a preset correction model to obtain a corrected image to be projected;
inputting the corrected image to be projected into a target vehicle-mounted head-up display system for projection display, wherein the preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and the target vehicle-mounted head-up display system in a training mode.
According to the image distortion correction method for the vehicle-mounted head-up display system, the image to be projected is obtained, the preset correction model is used for correcting the image to be projected to obtain the corrected image to be projected, the corrected image to be projected is input into the target vehicle-mounted head-up display system to be projected and displayed, and the preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and training of the target vehicle-mounted head-up display system. Therefore, before the image is projected and displayed, the image is corrected by the correction model, the image is input into the vehicle-mounted head-up display system to be projected and displayed after the image is corrected, and the distortion is corrected by the correction model.
This application another aspect embodiment provides a vehicle-mounted head-up display system image distortion correcting device, includes:
the first acquisition module is used for acquiring an image to be projected;
the correction module is used for correcting the image to be projected by using a preset correction model to obtain a corrected image to be projected;
and the display module is used for inputting the corrected image to be projected into a target vehicle-mounted head-up display system for projection display, wherein the preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and the training of the target vehicle-mounted head-up display system.
The on-vehicle head-up display system image distortion correcting device of this application embodiment utilizes predetermined correction model to treat the image of projection and corrects the processing through acquireing the image of treating the projection, obtains the image of treating the projection after correcting, will correct the image input target on-vehicle head-up display system of treating the projection after and throw the display, and wherein, predetermined correction model is for utilizing the training data set that corresponds with the on-vehicle head-up display system of target and the on-vehicle head-up display system training of target to generate. Therefore, before the image is projected and displayed, the image is corrected by the correction model, the image is input into the vehicle-mounted head-up display system to be projected and displayed after the image is corrected, and the distortion is corrected by the correction model.
Another embodiment of the present application provides a computer device, including a processor and a memory;
wherein the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the image distortion correction method of the vehicle-mounted head-up display system according to the embodiment of the above aspect.
Another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the image distortion correction method of the vehicle-mounted head-up display system as described in the above one embodiment.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 is a schematic flowchart of an image distortion correction method for a vehicle-mounted head-up display system according to an embodiment of the present disclosure;
FIG. 2 is a schematic flowchart illustrating another image distortion correction method for a vehicle-mounted head-up display system according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of another image distortion correction method for a vehicle-mounted head-up display system according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of an image distortion correction device of a vehicle-mounted head-up display system according to an embodiment of the present disclosure;
FIG. 5 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.
An image distortion correction method and apparatus of a vehicle-mounted head-up display system according to an embodiment of the present application will be described below with reference to the drawings.
The embodiment of the application provides an image distortion correction method for a vehicle-mounted head-up display system, aiming at the problems of high design difficulty and high cost of a method for performing distortion correction by adding an optical element in the related art.
Fig. 1 is a schematic flow chart of an image distortion correction method for a vehicle-mounted head-up display system according to an embodiment of the present application.
The image distortion correction method for the vehicle-mounted head-up display system can be executed through the image distortion correction device for the vehicle-mounted head-up display system, and the device can be configured in a computer device, so that the image to be projected is corrected and then projected and displayed through a preset correction model, and the image to be projected is subjected to distortion correction.
As shown in fig. 1, the image distortion correction method for the vehicle-mounted head-up display system includes:
In this embodiment, the vehicle-mounted head-up display system mounted on the vehicle may perform projection display on a video, such as a movie or an image, and then the image to be projected may be a video frame image or a single image.
And 102, correcting the image to be projected by using a preset correction model to obtain a corrected image to be projected.
The preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and training of the target vehicle-mounted head-up display system and is used for correcting the image so as to enable the corrected image to be distorted. Wherein the training dataset comprises a large number of images.
After the image to be projected is obtained, the image to be projected is corrected by using a preset correction model, and the corrected image to be projected is obtained. Here, the corrected image to be projected is an image with distortion.
And 103, inputting the corrected image to be projected into a target vehicle-mounted head-up display system for projection display.
After the corrected image to be projected is obtained, the corrected image to be projected is input into a target vehicle-mounted head-up display system for projection display, and the image displayed by projection is less distorted or has no distortion after passing through a front windshield, so that the projection display quality is greatly improved.
When the video is subjected to projection display, each frame of image can be corrected by using a preset correction model, and the corrected image is input into a target vehicle-mounted head-up display system for projection display. Therefore, when the user watches the video, the distortion of the projected and displayed picture is small or no distortion, and the watching experience of the user is improved.
Under normal conditions, if an image to be projected is directly input into a target vehicle-mounted head-up display system, the image is distorted by projection display due to the effect of a windshield. In this embodiment, an image to be projected is corrected by a preset correction model, so that the corrected image has distortion. When the corrected image to be projected is input to the target vehicle-mounted head-up display system and projected to the windshield, the corrected image is subjected to the distortion effect of the windshield, and finally the image displayed by projection is displayed normally.
In the related art, a discontinuous curved mirror or other optical elements are usually added in front of the windshield to compensate for the reduced imaging effect caused by the windshield, but the increase of the optical elements not only has great design difficulty, but also has high cost, and the volume of the vehicle-mounted head-up display system is larger.
In the embodiment of the application, when the target vehicle-mounted head-up display system is used for projecting and displaying the image to be projected, the image to be projected is corrected through the correction model, and then the corrected image to be projected is projected and displayed, so that the distortion correction is performed by using the correction model, the cost is low, the size of the vehicle-mounted head-up display system is reduced, and the parameters of a front windshield do not need to be considered.
Before the image to be projected is corrected by using the preset correction model, the preset correction model can be obtained through deep learning. Fig. 2 is a schematic flow chart of another image distortion correction method for a vehicle-mounted head-up display system according to an embodiment of the present application.
Before the image to be projected is corrected by using a preset correction model, as shown in fig. 2, the image distortion correction method for the vehicle-mounted head-up display system further includes:
The training data set comprises an input image set, and the input image set comprises a large number of images. When the training data set is acquired, a large number of images may be collected, and these images constitute the input image set, or the video is used as the input image set, and each frame of image in the video is the image in the input image set.
And 202, sequentially projecting and displaying each input image in the input image set by using the target vehicle-mounted head-up display system, and acquiring an output image corresponding to each input image.
In the embodiment, the position of the eyes of a person watching the projected image can be recorded when the target vehicle-mounted head-up display system projects, and then an image acquisition device such as a camera and the like is placed at the position of the eyes when the person watches the projected image.
In the target vehicle-mounted head-up display system, during the process of projection display of each input image in a set of input images in turn, a projection image corresponding to each input image is captured by an image capture device placed at the position of human eyes, and the projection image corresponding to each input image is referred to as an output image.
It will be appreciated that the output image of each input image is an image that is distorted.
And step 203, performing deep learning on each input image and the corresponding output image, and determining an image processing model corresponding to the target vehicle-mounted head-up display system.
After an output image corresponding to each input image in the input image set is obtained, an image processing model corresponding to the target vehicle-mounted head-up display system can be obtained through deep learning by utilizing each input image and the corresponding output image. Wherein the image processing model is operable to determine a relationship between the input image and the output image.
Specifically, an image processing model is obtained by performing deep learning on an initial image processing model using an input image in an input image set as a training sample. And in the training iterative process, calculating a loss function according to the image predicted by the model and the output image actually corresponding to the input image, and adjusting the model parameters according to the calculation result until the training is finished.
And step 204, determining a reference image corresponding to each input image according to the image processing model and each input image.
In the present embodiment, each output image is an image that is projected and displayed for each input image, the relationship between the input image and the output image can be determined according to the image processing model, and each output image is an image with distortion. Then, to make the input image projected by the in-vehicle head-up display system have no distortion, the input of the image processing model may be determined based on the image processing model and each input image, and the determined input of the image processing model is referred to as a reference image.
For example, P1 represents an input image, P2 represents an output image corresponding to the input image P1, C1Representing an image processing model, the relationship between an input image and a corresponding output image may be represented as P2 ═ C1(P1). Then, if the projected image is to be P1, i.e. the projected image is displayed normally, the modulo P1 is C1(P0), where P0 is the reference picture. Thus, the image P1 and the image processing model C are already input1In the case of (1), can be according to P1 and C1A reference image is determined.
In this embodiment, when the image to be displayed is a reference image, the image displayed by projection through the front windshield may be considered as a normal image. Therefore, in order to enable the image to be displayed in a normal projection manner, the image to be displayed should be processed to a certain extent so that the image to be displayed can be displayed normally.
After each input image and the corresponding reference image are obtained, the initial correction model can be trained by utilizing the input images, and the correction model corresponding to the target vehicle-mounted head-up display system is obtained. And in the training iterative process, calculating a loss function according to the image predicted by the initial correction model and a reference image actually corresponding to the input image, and adjusting the model parameters according to the calculation result until the training is finished.
Since the hardware parameters of the front windshield and the on-board head-up display system of each vehicle may not be completely consistent, the correction model corresponding to the on-board head-up display system of each vehicle can be obtained for different vehicles in the above manner.
In practical applications, the resolution of the image acquisition device used to acquire the output image may not coincide with the resolution of the input image. When the resolutions are not consistent, the resolution of the acquired output image may be inconsistent with that of the input image, and thus, an image processing model determined from the input image and the output image may be inaccurate, thereby causing an inaccuracy of a rectification model.
Therefore, before performing the deep learning on each input image and the corresponding output image to determine the image processing model corresponding to the target vehicle-mounted head-up display system, the first resolution of the image acquisition device may be determined to be consistent with the second resolution of the input image. And under the condition that the input image is determined to be consistent with the resolution of the image acquisition equipment, the resolution of the output image acquired by the image acquisition equipment is consistent with that of the input image, and the deep learning is carried out on each input image and each output image to obtain an image processing model.
In the embodiment of the application, before deep learning is carried out on each input image and the corresponding output image, the fact that the resolution ratio of the image acquisition equipment is consistent with that of the input image is determined, and when the resolution ratio is determined to be consistent, the deep learning is carried out on the input image and the corresponding output image, so that the accuracy of the correction model can be guaranteed.
In practical applications, the resolution of the input image may not be consistent with the resolution of the image acquisition device. How to determine the correction model when the resolution is not consistent is described below with reference to fig. 3. Fig. 3 is a schematic flowchart of another image distortion correction method for a vehicle-mounted head-up display system according to an embodiment of the present application.
Before the image to be projected is corrected by using a preset correction model, as shown in fig. 3, the image distortion correction method for the vehicle-mounted head-up display system further includes:
And 302, sequentially projecting and displaying each input image in the input image set by using the target vehicle-mounted head-up display system, and acquiring an output image corresponding to each input image.
In this embodiment, steps 301 to 302 are similar to steps 201 to 202 described above, and therefore are not described herein again.
If the resolution of the image acquisition device is inconsistent with the resolution of the input image, the resolution corresponding to the output image acquired by the image acquisition device is inconsistent with the resolution corresponding to the input image, and under the condition that the resolution of the image acquisition device is inconsistent with the resolution of the input image, the input image and the output image are subjected to deep learning, so that the accuracy of the obtained image processing model is low.
In order to ensure the accuracy of the image processing model, in this embodiment, when the first resolution of the image capturing device is not consistent with the second resolution corresponding to the input image, the scaling matrix between each output image and the corresponding input image is determined according to the first resolution of the image capturing device and the second resolution corresponding to each input image.
For example, the input image is P1, the output image corresponding to the input image P1 is P2, the scaling matrix between P1 and P2 is M, and the relationship between P1 and P2 can be expressed as P2 — M × P1.
Since the resolution between the input images in the input image set may be the same or different. Thus, a scaling matrix is determined between each input image in the set of input images and the corresponding output image. It will be appreciated that the scaling matrices for each pair of input images and output images may be the same or different.
And step 304, carrying out scaling processing on each input image according to the scaling matrix between each output image and the corresponding input image to obtain a scaled image.
In this embodiment, scaling processing is performed on each input image according to a scaling matrix between each output image and the corresponding input image, so that a scaled image of each input image can be obtained. And the resolution of the scaling image corresponding to each input image is consistent with the resolution of the image acquisition equipment.
For example, if the resolution corresponding to the input image is greater than the resolution of the image capturing device, the scaling matrix is used to scale the input image, and the resolution of the obtained scaled image is consistent with the resolution of the image capturing device. Thereby, the resolution of the output image acquired with the image acquisition device coincides with the resolution of the scaled image.
And 305, performing deep learning on each zoom image and the corresponding output image, and determining an image processing model corresponding to the target vehicle-mounted head-up display system.
In this embodiment, each zoomed image has the same resolution as the corresponding output image, and the image processing model corresponding to the target vehicle-mounted head-up display system is obtained by performing deep learning on each zoomed image and the corresponding output image. The specific process is the same as the process of performing the deep learning on the input image and the output image described in the above embodiment, and is not described herein again.
And 307, performing deep learning on the reference image and the input image corresponding to each input image, and determining a preset correction model.
In this embodiment, steps 306-307 are similar to steps 204-205, and thus are not described herein again.
In the embodiment of the application, when the first resolution of the image acquisition device is inconsistent with the second resolution corresponding to the input image, the scaling matrix between each output image and the corresponding input image is determined according to the first resolution of the image acquisition device and the second resolution corresponding to the input image, each input image is scaled by using the scaling matrix, each scaled image and the corresponding output image obtained by scaling are used for obtaining the image processing model, and therefore the accuracy of the image processing model and the accuracy of the correction model are guaranteed.
In practical application, a training data set corresponding to the target vehicle-mounted head-up display system can be obtained in a collection mode, and in order to improve efficiency, in an embodiment of the application, a preset image generation function can be used to generate the training data set corresponding to the target vehicle-mounted head-up display system.
Each input image and each output image can be regarded as a matrix, and each element value in the matrix corresponds to the gray value of each pixel point in the image, so that the image can be generated by using an image generation function, and the generated image is used as an image in an input image set.
In specific implementation, the number of images to be generated, the size of the images, the values of pixel points in the images and the like can be set through parameters in the image generation function. Thus, with the image generation function, a training data set can be acquired quickly.
In order to improve the accuracy of the image processing model, after the image processing model is obtained by using each input image and the corresponding output image, the trained image processing model can be tested by using a plurality of newly generated input images and corresponding output images, and whether the accuracy of the image processing model is greater than a threshold value or not is judged according to the calculation accuracy of the image predicted by the model and the actual output image.
When the accuracy of the image processing model is smaller than or equal to the threshold, the accuracy of the image processing model is considered to be not up to the requirement, and then the preset image generation function can be adjusted to generate a new training data set, so that the image processing model is continuously trained by utilizing the new training data set.
When the preset image generation function is adjusted, the value of the preset image generation function and/or the number of elements included in the preset image generation function may be adjusted. The value may refer to the number of generated images, or may be a value of a pixel point in an image.
In specific implementation, both the value and the contained quantity of the preset image generation function can be adjusted, and the value or the contained quantity can also be adjusted. During adjustment, the parameter values corresponding to the values and/or the contained quantities can be changed.
When the accuracy of the image processing model is greater than the threshold value, the accuracy of the image processing model can be considered to meet the requirement, and then the image processing model and each input image can be directly utilized to obtain a reference image corresponding to each input image, and then the input image and the reference image are utilized to obtain the correction model.
In the embodiment of the application, whether the accuracy of the image processing model is greater than the threshold value or not is judged, when the accuracy does not meet the requirement, a new training data set is obtained by adjusting the preset image generating function, the image processing model is trained continuously by using the new training data set, the accuracy of the image processing model can be improved, and the accuracy of the correction model is further improved.
In order to realize the above embodiments, the embodiment of the present application further provides an image distortion correction device for a vehicle-mounted head-up display system. Fig. 4 is a schematic structural diagram of an image distortion correction device of a vehicle-mounted head-up display system according to an embodiment of the present application.
As shown in fig. 4, the image distortion correcting apparatus for an in-vehicle head-up display system includes: a first acquisition module 410, a correction module 420, and a display module 430.
A first obtaining module 410, configured to obtain an image to be projected;
the correction module 420 is configured to perform correction processing on the image to be projected by using a preset correction model to obtain a corrected image to be projected;
and a display module 430, configured to input the corrected image to be projected into a target vehicle-mounted head-up display system for projection display, where a preset correction model is generated by using a training data set corresponding to the target vehicle-mounted head-up display system and training of the target vehicle-mounted head-up display system.
In a possible implementation manner of the embodiment of the present application, the apparatus may further include:
the second acquisition module is used for acquiring a training data set corresponding to the target vehicle-mounted head-up display system, wherein the training data set comprises an input image set;
the acquisition module is used for sequentially projecting and displaying each input image in the input image set by using the target vehicle-mounted head-up display system and acquiring an output image corresponding to each input image;
the first determining module is used for performing deep learning on each input image and the corresponding output image and determining an image processing model corresponding to the target vehicle-mounted head-up display system;
the second determining module is used for determining a reference image corresponding to each input image according to the image processing model and each input image;
and the third determining module is used for performing deep learning on the reference image and the input image corresponding to each input image and determining a preset correction model.
In a possible implementation manner of the embodiment of the present application, the acquisition module is specifically configured to: acquiring an output image corresponding to each input image by using image acquisition equipment;
the apparatus may further comprise:
and the fourth determining module is used for determining that the first resolution of the image acquisition equipment is consistent with the second resolution corresponding to the input image.
In a possible implementation manner of the embodiment of the application, if the first resolution of the image capturing device is different from the second resolution corresponding to the input image, the apparatus may further include:
the fifth determining module is used for determining a scaling matrix between each output image and the corresponding input image according to the first resolution of the image acquisition equipment and the second resolution corresponding to the input image;
the first determining module is specifically configured to:
carrying out scaling processing on each input image according to a scaling matrix between each output image and the corresponding input image to obtain a scaled image;
and performing deep learning on each zoom image and the corresponding output image to determine an image processing model corresponding to the target vehicle-mounted head-up display system.
In a possible implementation manner of the embodiment of the application, the second obtaining module is specifically configured to:
generating a training data set corresponding to the target vehicle-mounted head-up display system by using a preset image generation function;
the apparatus may further comprise:
the judging module is used for judging whether the accuracy of the image processing model is greater than a threshold value;
and the generating module is used for adjusting the preset image generating function to generate a new training data set when the accuracy of the image processing model is less than or equal to the threshold value.
In a possible implementation manner of the embodiment of the present application, the generating module is specifically configured to:
and adjusting the value of the preset image generation function and/or the number of the contained elements.
It should be noted that the above explanation of the embodiment of the image distortion correction method for the vehicle-mounted head-up display system is also applicable to the image distortion correction device for the vehicle-mounted head-up display system of the embodiment, and therefore, the explanation is omitted here.
The on-vehicle head-up display system image distortion correcting device of this application embodiment utilizes predetermined correction model to treat the image of projection and corrects the processing through acquireing the image of treating the projection, obtains the image of treating the projection after correcting, will correct the image input target on-vehicle head-up display system of treating the projection after and throw the display, and wherein, predetermined correction model is for utilizing the training data set that corresponds with the on-vehicle head-up display system of target and the on-vehicle head-up display system training of target to generate. Therefore, before the image is projected and displayed, the image is corrected by the correction model, the image is input into the vehicle-mounted head-up display system to be projected and displayed after the image is corrected, and the distortion is corrected by the correction model.
In order to implement the foregoing embodiments, an embodiment of the present application further provides a computer device, including a processor and a memory;
the processor reads executable program codes stored in the memory to run programs corresponding to the executable program codes, so as to realize the image distortion correction method of the vehicle-mounted head-up display system according to the embodiment.
FIG. 5 illustrates a block diagram of an exemplary computer device suitable for use in implementing embodiments of the present application. The computer device 12 shown in fig. 5 is only an example and should not bring any limitation to the function and scope of use of the embodiments of the present application.
As shown in FIG. 5, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally perform the functions and/or methodologies of the embodiments described herein.
The processing unit 16 executes various functional applications and data processing, for example, implementing the methods mentioned in the foregoing embodiments, by executing programs stored in the system memory 28.
In order to implement the above embodiments, the present application further proposes a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the image distortion correction method of the vehicle-mounted head-up display system as described in the above embodiments.
In the description of the present specification, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (12)
1. An image distortion correction method for a vehicle-mounted head-up display system is characterized by comprising the following steps:
acquiring an image to be projected;
correcting the image to be projected by using a preset correction model to obtain a corrected image to be projected, wherein the corrected image to be projected is an image with distortion;
inputting the corrected image to be projected into a target vehicle-mounted head-up display system for projection display, wherein the preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and the target vehicle-mounted head-up display system in a training mode;
before the image to be projected is corrected by using a preset correction model, the method further includes:
acquiring a training data set corresponding to the target vehicle-mounted head-up display system, wherein the training data set comprises an input image set;
sequentially projecting and displaying each input image in the input image set by using the target vehicle-mounted head-up display system, and acquiring an output image corresponding to each input image;
performing deep learning on each input image and the corresponding output image, and determining an image processing model corresponding to the target vehicle-mounted head-up display system;
determining a reference image corresponding to each input image according to the image processing model and each input image;
and carrying out deep learning on the reference image and the input image corresponding to each input image, and determining the preset correction model.
2. The method of claim 1, wherein said acquiring an output image corresponding to each input image comprises:
acquiring an output image corresponding to each input image by using image acquisition equipment;
before performing deep learning on each input image and the corresponding output image and determining the image processing model corresponding to the target vehicle-mounted head-up display system, the method further comprises the following steps:
determining that the first resolution of the image acquisition device is consistent with the corresponding second resolution of the input image.
3. The method as claimed in claim 2, wherein if the first resolution of the image capturing device is different from the second resolution corresponding to the input image, before performing the deep learning for each input image and the corresponding output image and determining the image processing model corresponding to the target vehicle-mounted head-up display system, the method further comprises:
determining a scaling matrix between each output image and the corresponding input image according to the first resolution of the image acquisition equipment and the second resolution corresponding to the input image;
the deep learning of each input image and the corresponding output image is carried out, and the image processing model corresponding to the target vehicle-mounted head-up display system is determined, and the method comprises the following steps:
carrying out scaling processing on each input image according to a scaling matrix between each output image and the corresponding input image to obtain a scaled image;
and performing deep learning on each zoom image and the corresponding output image, and determining an image processing model corresponding to the target vehicle-mounted head-up display system.
4. The method as recited in claim 1, wherein said obtaining a training data set corresponding to said target in-vehicle heads-up display system comprises:
generating a training data set corresponding to the target vehicle-mounted head-up display system by using a preset image generation function;
after the deep learning is carried out on each input image and the corresponding output image and the image processing model corresponding to the target vehicle-mounted head-up display system is determined, the method further comprises the following steps:
judging whether the accuracy of the image processing model is greater than a threshold value;
if not, adjusting the preset image generation function to generate a new training data set.
5. The method of claim 4, wherein said adjusting said preset image generation function comprises:
and adjusting the value of the preset image generation function and/or the number of elements contained in the preset image generation function.
6. An image distortion correction apparatus for an on-vehicle head-up display system, comprising:
the first acquisition module is used for acquiring an image to be projected;
the correction module is used for correcting the image to be projected by using a preset correction model to obtain a corrected image to be projected, wherein the corrected image to be projected is an image with distortion;
the display module is used for inputting the corrected image to be projected into a target vehicle-mounted head-up display system for projection display, wherein the preset correction model is generated by utilizing a training data set corresponding to the target vehicle-mounted head-up display system and the target vehicle-mounted head-up display system in a training mode;
the second acquisition module is used for acquiring a training data set corresponding to the target vehicle-mounted head-up display system, wherein the training data set comprises an input image set;
the acquisition module is used for sequentially projecting and displaying each input image in the input image set by using the target vehicle-mounted head-up display system and acquiring an output image corresponding to each input image;
the first determining module is used for performing deep learning on each input image and the corresponding output image and determining an image processing model corresponding to the target vehicle-mounted head-up display system;
the second determining module is used for determining a reference image corresponding to each input image according to the image processing model and each input image;
and the third determining module is used for performing deep learning on the reference image and the input image corresponding to each input image and determining the preset correction model.
7. The apparatus of claim 6, wherein the acquisition module is specifically configured to: acquiring an output image corresponding to each input image by using image acquisition equipment;
the device further comprises:
and the fourth determination module is used for determining that the first resolution of the image acquisition equipment is consistent with the second resolution corresponding to the input image.
8. The apparatus of claim 7, wherein if the first resolution of the image capture device is different from the second resolution corresponding to the input image, the apparatus further comprises:
a fifth determining module, configured to determine a scaling matrix between each output image and the corresponding input image according to the first resolution of the image capturing device and the second resolution corresponding to the input image;
the first determining module is specifically configured to:
carrying out scaling processing on each input image according to a scaling matrix between each output image and the corresponding input image to obtain a scaled image;
and performing deep learning on each zoom image and the corresponding output image, and determining an image processing model corresponding to the target vehicle-mounted head-up display system.
9. The apparatus of claim 6, wherein the second obtaining module is specifically configured to:
generating a training data set corresponding to the target vehicle-mounted head-up display system by using a preset image generation function;
the device further comprises:
the judging module is used for judging whether the accuracy of the image processing model is greater than a threshold value;
and the generating module is used for adjusting the preset image generating function to generate a new training data set when the accuracy of the image processing model is less than or equal to a threshold value.
10. The apparatus of claim 9, wherein the generating module is specifically configured to:
and adjusting the value of the preset image generation function and/or the number of elements contained in the preset image generation function.
11. A computer device comprising a processor and a memory;
wherein the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory for implementing the image distortion correction method of the vehicle-mounted head-up display system according to any one of claims 1 to 5.
12. A computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the image distortion correction method of the on-vehicle head-up display system according to any one of claims 1 to 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910270514.1A CN109993713B (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
| CN202111204135.6A CN114037620A (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910270514.1A CN109993713B (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111204135.6A Division CN114037620A (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109993713A CN109993713A (en) | 2019-07-09 |
| CN109993713B true CN109993713B (en) | 2021-11-05 |
Family
ID=67132386
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910270514.1A Active CN109993713B (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
| CN202111204135.6A Withdrawn CN114037620A (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111204135.6A Withdrawn CN114037620A (en) | 2019-04-04 | 2019-04-04 | Image distortion correction method and device for vehicle-mounted head-up display system |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN109993713B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110942434B (en) * | 2019-11-22 | 2023-05-05 | 华兴源创(成都)科技有限公司 | Display compensation system and method of display panel |
| CN111141492B (en) * | 2019-12-13 | 2021-11-02 | 中国航空工业集团公司洛阳电光设备研究所 | Head-up display system ray apparatus calibration stand |
| CN112995620B (en) * | 2019-12-17 | 2024-01-02 | 青岛海高设计制造有限公司 | Method for correcting cylindrical projection, device for cylindrical projection and household appliance |
| CN111260586B (en) * | 2020-01-20 | 2023-07-04 | 北京百度网讯科技有限公司 | Correction method and device for distorted document image |
| CN113746999B (en) * | 2020-05-15 | 2023-01-03 | 华为技术有限公司 | Imaging method, imaging device, optical imaging system and vehicle |
| CN113628282A (en) * | 2021-08-06 | 2021-11-09 | 深圳市道通科技股份有限公司 | Pattern projection correction apparatus, method, and computer-readable storage medium |
| CN115115552B (en) * | 2022-08-25 | 2022-11-18 | 腾讯科技（深圳）有限公司 | Image correction model training method, image correction device and computer equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202075492U (en) * | 2011-06-15 | 2011-12-14 | 陈嘉华 | Image information issuing system for vehicle rear window |
| CN105404006A (en) * | 2015-12-24 | 2016-03-16 | 深圳点石创新科技有限公司 | Head up display |
| CN107527324A (en) * | 2017-07-13 | 2017-12-29 | 江苏泽景汽车电子股份有限公司 | A kind of pattern distortion antidote of HUD |
| CN107749048A (en) * | 2017-07-31 | 2018-03-02 | 中国银联股份有限公司 | Image flame detection system and method, pseudoisochromatic plate are as correction system and method |
| CN108053373A (en) * | 2017-12-05 | 2018-05-18 | 长沙全度影像科技有限公司 | One kind is based on deep learning model fisheye image correcting method |
| CN108171673A (en) * | 2018-01-12 | 2018-06-15 | 京东方科技集团股份有限公司 | Image processing method, device, vehicle-mounted head-up-display system and vehicle |
| JP6390032B2 (en) * | 2015-09-02 | 2018-09-19 | カルソニックカンセイ株式会社 | Head-up display distortion correction method and head-up display distortion correction apparatus using the same |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102663734A (en) * | 2012-03-15 | 2012-09-12 | 天津理工大学 | Fish eye lens calibration and fish eye image distortion correction method |
| CN108876725A (en) * | 2017-05-12 | 2018-11-23 | 深圳市魔眼科技有限公司 | A kind of virtual image distortion correction method and system |
| CN108470328A (en) * | 2018-03-28 | 2018-08-31 | 百度在线网络技术（北京）有限公司 | Method and apparatus for handling image |
| CN109034017B (en) * | 2018-07-12 | 2021-05-14 | 北京华捷艾米科技有限公司 | Head pose estimation method and machine readable storage medium |
-
2019
- 2019-04-04 CN CN201910270514.1A patent/CN109993713B/en active Active
- 2019-04-04 CN CN202111204135.6A patent/CN114037620A/en not_active Withdrawn
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202075492U (en) * | 2011-06-15 | 2011-12-14 | 陈嘉华 | Image information issuing system for vehicle rear window |
| JP6390032B2 (en) * | 2015-09-02 | 2018-09-19 | カルソニックカンセイ株式会社 | Head-up display distortion correction method and head-up display distortion correction apparatus using the same |
| CN105404006A (en) * | 2015-12-24 | 2016-03-16 | 深圳点石创新科技有限公司 | Head up display |
| CN107527324A (en) * | 2017-07-13 | 2017-12-29 | 江苏泽景汽车电子股份有限公司 | A kind of pattern distortion antidote of HUD |
| CN107749048A (en) * | 2017-07-31 | 2018-03-02 | 中国银联股份有限公司 | Image flame detection system and method, pseudoisochromatic plate are as correction system and method |
| CN108053373A (en) * | 2017-12-05 | 2018-05-18 | 长沙全度影像科技有限公司 | One kind is based on deep learning model fisheye image correcting method |
| CN108171673A (en) * | 2018-01-12 | 2018-06-15 | 京东方科技集团股份有限公司 | Image processing method, device, vehicle-mounted head-up-display system and vehicle |
Non-Patent Citations (3)
| Title |
|---|
| "Automatic Distortion Correction for a Full Windshield Head-Up Display System";Fabian Dominik Blaicher;《KIT》;20090130;第1-23页 * |
| "基于计算机视觉算法的图像处理技术";刘海玲;《计算机与数字工程》;20190331;第47卷(第3期);第672-677页 * |
| "车载平视显示技术";王兴等;《电光与控制》;20140131;第21卷(第1期);第55-58页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114037620A (en) | 2022-02-11 |
| CN109993713A (en) | 2019-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109993713B (en) | Image distortion correction method and device for vehicle-mounted head-up display system | |
| CN110969580B (en) | Image display method and device | |
| CN110263732B (en) | Multi-scale target detection method and device | |
| CN109873997A (en) | Projected picture correcting method and device | |
| CN110618803B (en) | Image display method and device | |
| CN110070623B (en) | Guide line drawing prompting method, device, computer equipment and storage medium | |
| CN112614192A (en) | On-line calibration method of vehicle-mounted camera and vehicle-mounted information entertainment system | |
| CN113672077A (en) | Data processing method and equipment thereof | |
| KR20140117876A (en) | Method and apparatus for correcting distortion of image | |
| CN113807160B (en) | Method and device for detecting hijacking of image acquisition equipment and computer equipment | |
| JP2008177742A (en) | Camera parameter acquisition device | |
| KR102076635B1 (en) | Apparatus and method for generating panorama image for scattered fixed cameras | |
| CN116862788A (en) | CMS field checking method, system, device and storage medium | |
| CN114821544B (en) | Perception information generation method and device, vehicle, electronic equipment and storage medium | |
| CN113837987B (en) | Tongue image acquisition method and device and computer equipment | |
| CN115205138A (en) | Method and system for correcting image distortion | |
| CN114418908A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN115578283B (en) | Distortion correction method and device for HUD imaging, terminal equipment and storage medium | |
| US20240112307A1 (en) | Image processing device and image display device | |
| CN113063432B (en) | Visible light visual navigation method in smoke environment | |
| CN115866400A (en) | Anti-shake control method and device and electronic equipment | |
| CN111126105B (en) | Human body key point detection method and device | |
| CN115471406A (en) | Image distortion correction method, apparatus, device and medium for head-up display apparatus | |
| JP2007116332A (en) | Image processing apparatus | |
| CN118134944A (en) | Image processing method, device, terminal equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211012 Address after: 100176 101, floor 1, building 1, yard 7, Ruihe West 2nd Road, Beijing Economic and Technological Development Zone, Daxing District, Beijing Applicant after: Apollo Zhilian (Beijing) Technology Co.,Ltd. Address before: 100085 Baidu Building, 10 Shangdi Tenth Street, Haidian District, Beijing Applicant before: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |