CN116433512A - Image processing method, device and storage medium - Google Patents
Image processing method, device and storage medium Download PDFInfo
- Publication number
- CN116433512A CN116433512A CN202310294732.5A CN202310294732A CN116433512A CN 116433512 A CN116433512 A CN 116433512A CN 202310294732 A CN202310294732 A CN 202310294732A CN 116433512 A CN116433512 A CN 116433512A
- Authority
- CN
- China
- Prior art keywords
- image
- objects
- glass mirror
- image processing
- different angles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 29
- 239000011521 glass Substances 0.000 claims abstract description 60
- 238000012937 correction Methods 0.000 claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 22
- 230000003321 amplification Effects 0.000 claims abstract description 21
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000009466 transformation Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 abstract description 8
- 230000008859 change Effects 0.000 description 13
- 238000013461 design Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an image processing method, an image processing device and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: distortion correction is carried out on the photographed distorted image obtained by the fisheye camera, and corrected images with consistent amplification coefficients are obtained; correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image; the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers. The image processing method, the device and the storage medium can correct the distorted image obtained by the fisheye camera on the premise that the calculation power of the vehicle-mounted electronic vision mirror system is limited.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image processing method, an image processing device and a storage medium.
Background
In order to ensure driving safety, according to the standard of the ISO (digital camera sensitivity quantitative regulation), the object imaging displayed in the central area in the image of the automobile electronic rearview mirror needs to be deformed less, and the display area extracts a sufficiently large field of view so as to enable a driver to accurately judge the driving environment, thereby providing conditions for driving pre-judgment.
At present, most of electronic rearview mirrors for automobiles on the market use fisheye cameras, which cannot achieve the display effect specified by the ISO standard. In order to obtain a better look and feel, the current common method is to correct the distorted image obtained by the automobile electronic rearview mirror through an acceptance or VGGNet (Visual Geometry Group Networks) deep learning network, and the transformation mode can improve the accuracy of correcting the distorted image, but the method cannot be used in a vehicle-mounted electronic rearview mirror system with limited calculation power due to large calculation amount of a deep learning algorithm.
Therefore, how to provide an effective solution to correct the distorted image obtained by the fisheye camera under the condition that the calculation power of the vehicle-mounted electronic vision mirror system is limited has become a urgent problem in the prior art.
Disclosure of Invention
An object of the present invention is to provide an image processing method, apparatus and storage medium, which are used to solve the above-mentioned problems in the prior art.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
in a first aspect, the present invention provides an image processing method for correcting a distorted image captured by a fisheye camera on an automobile, including:
distortion correction is carried out on the photographed distorted image obtained by the fisheye camera, and corrected images with consistent amplification coefficients are obtained;
correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image;
the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers.
In one possible design, the correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficient of the glass mirror for the objects with different angles includes:
correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror on objects with different angles, polar meridians corresponding to projection points of each pixel point in the corrected image on a plane coordinate system and included angles between the polar meridians corresponding to each pixel point in the corrected image and an x-axis in the plane coordinate system;
the plane coordinate system is established by taking a parallel vehicle body direction as a y axis, a vertical vehicle body direction as an x axis and a point projected on a horizontal plane by the fisheye camera as an origin.
In one possible design, the correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficients of the glass mirror for objects with different angles, the polar channel corresponding to the projection point of each pixel point in the corrected image on the plane coordinate system, and the included angle between the polar channel corresponding to each pixel point in the corrected image and the x-axis in the plane coordinate system includes:
correcting pixel coordinates of each pixel point in the corrected image according to the following formula;
wherein m is 00 、m 01 、m 10 、m 11 、m 20 And m 21 The parameters in the internal reference matrix of the fisheye camera are k as a constant, r represents the image under a plane coordinate system established by taking the parallel vehicle body direction as the y axis, the vertical vehicle body direction as the x axis and the point of the fisheye camera projected on the horizontal plane as the originThe polar channel corresponding to the projection point of any pixel point on the plane coordinate system, theta represents the included angle between the polar channel corresponding to any pixel point and the x-axis, M (θ) The observation magnification coefficient of the object with the polar angle theta of the glass mirror under the plane coordinate system is shown.
In one possible design, the observation magnification factor is M (θ) ＝-0.6(π/2+β 0 -θ) 2 +0.001(π/2+β 0 - θ) +0.1, where β 0 And the included angle between the projection line of sight of the driver and the y axis is shown, and the projection line of sight of the driver is a connecting line formed by the projection of the eye point of the driver on the plane coordinate system and the origin.
In one possible design, before correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficient of the glass mirror for the object at different angles, the method further includes:
and establishing a plane coordinate system by taking a parallel vehicle body direction as a y axis, a vertical vehicle body direction as an x axis and a point projected on a horizontal plane by the fisheye camera as an origin.
In one possible design, the correcting distortion of the distorted image captured by the fisheye camera includes:
and carrying out distortion correction on the distorted image shot by the fisheye camera by using a Zhengyou plane calibration method and a polynomial model transformation algorithm.
In a second aspect, the present invention provides an image processing apparatus for correcting a distorted image captured by a fisheye camera on an automobile, comprising:
the distortion correcting unit is used for carrying out distortion correction on the distorted image shot by the fisheye camera to obtain corrected images with consistent amplification coefficients at all positions;
the coordinate correction unit is used for correcting pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image;
the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers.
In a third aspect, the present invention provides an image processing apparatus comprising a memory, a processor and a transceiver in communication with each other in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to receive and transmit a message, and the processor is configured to read the computer program and perform the image processing method according to the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon which, when executed on a computer, perform the image processing method of the first aspect.
In a fifth aspect, the present invention provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the image processing method according to the first aspect.
The beneficial effects are that:
the invention creatively provides an image processing scheme, namely, distortion correction is carried out on a distorted image obtained by a fisheye camera, and a corrected image with consistent amplification coefficients is obtained; correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image; the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers. The distortion image captured by the fisheye camera is subjected to distortion correction firstly, the consistency of the amplification coefficients of the image is ensured, so that the amplification coefficients of the image are close to those of the glass mirror, then the pixel coordinates of each pixel point in the corrected image are corrected based on the observation amplification coefficients of the glass mirror on objects with different angles, so that the corrected image is more similar to the image directly observed by naked eyes, a better impression effect is achieved, the driving safety is greatly improved, complex operation is not required in the process, and the distortion image obtained by the fisheye camera can be corrected on the premise that the calculation power of a vehicle-mounted electronic vision mirror system is limited, so that the practical application and popularization are convenient.
Drawings
Fig. 1 is a flowchart of an image processing method provided in an embodiment of the present application;
fig. 2 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present application.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly described below with reference to the accompanying drawings and the description of the embodiments or the prior art, and it is obvious that the following description of the structure of the drawings is only some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art. It should be noted that the description of these examples is for aiding in understanding the present invention, but is not intended to limit the present invention.
In order to solve the problem that in the prior art, the calculation power of the vehicle-mounted electronic vision mirror system is limited and cannot be used for correcting the distorted image acquired by the fisheye camera, the embodiment of the application provides an image processing method, an image processing device and a storage medium.
The image processing method provided in the embodiment of the present application may be applied to a vehicle-mounted electronic mirror system, and the image processing method provided in the embodiment of the present application will be described in detail below, where it is understood that the execution subject does not limit the embodiment of the present application.
As shown in fig. 1, there is a flowchart of an image processing method provided in the first aspect of the embodiment of the present application, and the image processing method may, but is not limited to, include the following steps S101 to S102.
S101, carrying out distortion correction on a distorted image shot by a fisheye camera to obtain corrected images with consistent amplification coefficients.
In the embodiment of the application, when correcting the distortion image shot by the fisheye camera, the distortion correction can be performed on the distortion image shot by the fisheye camera to obtain the corrected image with consistent amplification coefficients.
And when the distortion image shot by the fisheye camera is subjected to distortion correction, the distortion image shot by the fisheye camera can be subjected to distortion correction through a Zhang Zhengyou plane calibration method and a polynomial model transformation algorithm, so that corrected images with consistent amplification coefficients are obtained, and the corrected images are close to images observed through a glass mirror. Wherein, the Zhang Zhengyou plane calibration method and the polynomial model transformation algorithm are both the prior art, the principles thereof are not described in detail in the embodiments of the present application,
it will be appreciated that in other embodiments, the distorted image may be distorted by other existing methods, which are not illustrated in the embodiments of the present application.
In addition, it should be noted that the method provided in the embodiment of the present application may also be applied to other types of cameras that are mounted on a vehicle and are similar to fish-eye imaging in that the lens protrudes toward the front of the lens, which causes distortion in image transmission.
S102, correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror to objects with different angles, and obtaining a corrected image.
The distortion correction is carried out on the distorted image shot by the fisheye camera through a Zhengyou plane calibration method and a polynomial model transformation algorithm, the obtained corrected image is close to the image observed through the glass mirror, and the image observed through the glass mirror has a certain difference from the image observed through naked eyes. Based on this, after distortion correction is performed on the distorted image, the embodiment of the application may further correct pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficients of the glass mirror on the objects with different angles, so as to obtain a corrected image.
The observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers.
Specifically, when correcting the pixel coordinates of each pixel point in the corrected image, the pixel coordinates of each pixel point in the corrected image may be corrected based on the observation magnification coefficients of the glass mirror for objects at different angles, the polar meridians corresponding to the projection points of each pixel point in the corrected image on the plane coordinate system, and the included angles between the polar meridians corresponding to each pixel point in the corrected image and the x-axis in the plane coordinate system. The plane coordinate system is established by taking a parallel vehicle body direction as a y axis, a vertical vehicle body direction as an x axis and a point projected on a horizontal plane by the fisheye camera as an origin.
More specifically, the pixel coordinates of each pixel point in the corrected image may be corrected according to the following formula (1):
wherein m is 00 、m 01 、m 10 、m 11 、m 20 And m 21 The parameters in the internal reference matrix of the fisheye camera are k as a constant, r represents the polar distance corresponding to the projection point of any pixel point in the image on the plane coordinate system under the plane coordinate system established by taking the parallel vehicle body direction as the y axis, the vertical vehicle body direction as the x axis and the point of the fisheye camera projected on the horizontal plane as the origin, θ represents the included angle between the polar distance corresponding to any pixel point and the x axis, M (θ) The observation magnification coefficient of the object with the polar angle theta of the glass mirror under the plane coordinate system is shown.
In the embodiment of the application, the reference matrix of the fisheye camera can be expressed as
The internal reference matrix is the fish-eye cameraParameters are known. The observation magnification factor can be expressed as M (θ) ＝-0.6(π/2+β 0 -θ) 2 +0.001(π/2+β 0 - θ) +0.1, where β 0 And the included angle between the projection line of sight of the driver and the y axis is shown, and the projection line of sight of the driver is a connecting line formed by the projection of the eye point of the driver on the plane coordinate system and the origin.
In one or more embodiments, before correcting the pixel coordinates of each pixel in the corrected image based on the observed magnification factor of the glass mirror for the object at the different angles, the method further comprises: and establishing a plane coordinate system by taking a parallel vehicle body direction as a y axis, a vertical vehicle body direction as an x axis and a point projected on a horizontal plane by the fisheye camera as an origin.
Next, a deduction description will be made on a calibration formula for correcting pixel coordinates of each pixel point in the corrected image in the embodiment of the present application.
In the embodiment of the application, the correction formula for correcting the image can be determined by researching the relevant influence factors of the glass mirror and the magnification factor.
Taking the driver side wide-angle external view mirror as an example, assuming that O is the position of the glass mirror, and the projection of the glass mirror on the ground is the same as the projection position of the fisheye camera on the ground, a is the position of the object, B is the driver's eye point, and the magnification factor of the glass mirror is mainly related to the distance d (length of OA, hereinafter referred to as the object distance d) from the object to the glass mirror and the included angle β (hereinafter referred to as the driver's field angle) between the eye point and the connecting line OB of the glass mirror according to the ISO standard.
In the ISO standard, the amplification factor is calculated as follows:
(1) When the object distance d is infinity and the driver viewing angle β=0 (i.e., located directly in front of the glass mirror), the corresponding magnification factor calculation formula is shown as the following formula (2).
Wherein r is mirror Representing the radius of curvature of the glass mirror, a mirror Indicating eyepoint to glassDistance of the mirror.
(2) When the object distance is infinity and the driver's view angle β is changed, the corresponding magnification coefficient is shown in the following formula (3).
Wherein a=a mirror ，r＝r mirror Δa represents a value of a change in the distance from the eyepoint to the glass mirror, Δβ represents a value of a change in the driver's view angle β, β1 represents a value before the change in the driver's view angle β, β2 represents a value after the change in the driver's view angle β, and β2 > β1.
When the driver viewing angle β=0, the corresponding magnification factor M (β=0, d) when the object distance d is changed is as shown in formula (4).
The combination of formulas (2), (3) and (4) shows that the amplification factor formulas for changing β and d are shown in formula (5).
From equation (5), the magnification factor of the glass mirror is a function of the object distance d and the driver viewing angle β. In actual driving, the driver generally focuses on the environmental information within 30m from the vehicle in the main rearview mirror observation, and the observed viewing angle is generally 55 ° to 75 °. As shown in table 1 below, the rate of change of the magnification factor M and the ratio thereof, which are caused when the driver-side view angle β constant object distance d is changed and when the object distance d constant view angle β is changed, are calculated according to the formula (5) in a test.
TABLE 1
| Constant beta, change d | d constant, beta variation | (d constant, beta variation)/(beta constant, d variation) | |
| Main rearview mirror | 10.71％ | 43.97％ | 4.11 |
| Wide-angle external view mirror | 20.43％ | 128.25％ | 6.28 |
As is clear from table 1, taking the driver-side wide-angle outside mirror as an example, the influence of the driver's field angle β change on the magnification factor is 6.28 times the influence of the object distance d change on the magnification factor, and thus the influence of the driver's field angle β change on the magnification factor can be taken as a main subject.
In the embodiment of the application, a plurality of curves under different object distances can be fitted through a least square method, and the fitted amplification coefficient can be expressed as M (beta) = -0.6beta 2 +0.001β+0.1。
The analysis of the magnification factor of the glass mirror is a transformation function obtained under the top view, and cannot be directly applied to the transformation of the image magnification factor of the electronic rearview mirror (fish-eye camera), and the relation between the magnification factor M under the top view and the view angle β of the driver needs to be transformed into the relation between the pixels on the display image by projection transformation. From analysis of the transformation function, it is known that the image magnification factor transformation is actually scaling in the polar radial direction with the change in magnification factor while the polar angle of the image remains unchanged in the polar coordinates of the top view.
Therefore, in the embodiment of the application, a polar coordinate with the perpendicular point O of the glass mirror and the ground as the origin can be constructed on the top view of the automobile, the polar diameter is r, the polar angle is θ, the parallel automobile body is the y axis, and the vertical automobile body is the x axis. Let the included angle between the driver's eye point and the origin O line and the y-axis be beta 0 The relationship between the amplification factor M and the polar angle θ can be obtained as:
M (θ) ＝-0.6(π/2+β 0 -θ) 2 +0.001(π/2+β 0 -θ)+0.1 (6)
the change of the magnification coefficient in the polar coordinate can look at the change of the relative position of the object and the origin, namely the length of the polar path r is transformed, and the polar path after the change is r', the following formula (7) is provided.
r′＝kM (θ) r (7)
Where k is a constant, let θ=0, km (θ) As shown in the combination formula (7) =1, the relationship between r and r' is shown in the formula (8).
The coordinates of the center point of the object in the polar coordinates have the following relation (9).
The polar coordinate system is converted into a Cartesian coordinate system, and the coordinates after the point transformation on the image are given as (x ', y'), and the following formula (10) is given.
However, the image relationship obtained under the top view cannot be directly used for the image shot by the camera, and the relationship of the top view is converted into the relationship of the pixel points on the image shot by the electronic rearview mirror through projection conversion.
The following formula (11) can be obtained according to the image procedure of the fisheye camera.
In the method, in the process of the invention,
representing pixel coordinates, +.>
Representing the coordinates of a point on the world coordinate system,/-, for example>
Is an internal reference matrix of the fish-eye camera.
Assuming that the pixel coordinates of the changed image point are (u ', v'), the pixel coordinates of the changed point can be expressed as:
In summary, according to the image processing method provided by the embodiment of the application, distortion correction is performed on the distorted image shot by the fisheye camera, so that a corrected image with consistent amplification coefficients is obtained; then correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image; the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers. The distortion image captured by the fisheye camera is subjected to distortion correction firstly, the consistency of the amplification coefficients of the image is ensured, so that the amplification coefficients of the image are close to those of the glass mirror, then the pixel coordinates of each pixel point in the corrected image are corrected based on the observation amplification coefficients of the glass mirror on objects with different angles, so that the corrected image is more similar to the image directly observed by naked eyes, a better impression effect is achieved, the driving safety is greatly improved, complex operation is not required in the process, and the distortion image obtained by the fisheye camera can be corrected on the premise that the calculation power of a vehicle-mounted electronic vision mirror system is limited, so that the practical application and popularization are convenient.
Referring to fig. 2, a second aspect of the embodiments of the present application provides an image processing apparatus for correcting a distorted image captured by a fisheye camera on an automobile, the image processing apparatus comprising:
the distortion correcting unit is used for carrying out distortion correction on the distorted image shot by the fisheye camera to obtain corrected images with consistent amplification coefficients at all positions;
the coordinate correction unit is used for correcting pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image;
the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers.
The working process, working details and technical effects of the device provided in the second aspect of the present embodiment may be referred to in the first aspect of the present embodiment, and are not described herein.
As shown in fig. 3, a third aspect of the embodiment of the present application provides another image processing apparatus, including a memory, a processor, and a transceiver, which are sequentially communicatively connected, where the memory is configured to store a computer program, the transceiver is configured to send and receive a message, and the processor is configured to read the computer program, and perform the image processing method according to the first aspect of the embodiment.
By way of specific example, the Memory may include, but is not limited to, random Access Memory (RAM), read Only Memory (ROM), flash Memory (Flash Memory), first-in-first-out Memory (FIFO), and/or first-in-last-out Memory (FILO), etc.; the processor may not be limited to a processor adopting architecture such as a microprocessor, ARM (Advanced RISC Machines), X86, etc. of the model STM32F105 series or a processor integrating NPU (neural-network processing units); the transceiver may be, but is not limited to, a WiFi (wireless fidelity) wireless transceiver, a bluetooth wireless transceiver, a general packet radio service technology (General Packet Radio Service, GPRS) wireless transceiver, a ZigBee protocol (low power local area network protocol based on the ieee802.15.4 standard), a 3G transceiver, a 4G transceiver, and/or a 5G transceiver, etc.
A fourth aspect of the present embodiment provides a computer readable storage medium storing instructions comprising the image processing method according to the first aspect of the present embodiment, i.e. the computer readable storage medium has instructions stored thereon, which when executed on a computer, perform the image processing method according to the first aspect. The computer readable storage medium refers to a carrier for storing data, and may include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash Memory, and/or a Memory Stick (Memory Stick), etc., where the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable devices.
A fifth aspect of the present embodiment provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the image processing method according to the first aspect of the embodiment, wherein the computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus.
It should be understood that specific details are provided in the following description to provide a thorough understanding of the example embodiments. However, it will be understood by those of ordinary skill in the art that the example embodiments may be practiced without these specific details. For example, a system may be shown in block diagrams in order to avoid obscuring the examples with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the example embodiments.
Finally, it should be noted that: the foregoing description is only of the preferred embodiments of the invention and is not intended to limit the scope of the invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (9)
1. An image processing method for correcting a distorted image captured by a fisheye camera on an automobile, comprising:
distortion correction is carried out on the photographed distorted image obtained by the fisheye camera, and corrected images with consistent amplification coefficients are obtained;
correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image;
the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers.
2. The image processing method according to claim 1, wherein the correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficient of the glass mirror for the object at the different angles includes:
correcting pixel coordinates of each pixel point in the corrected image based on observation magnification coefficients of the glass mirror on objects with different angles, polar meridians corresponding to projection points of each pixel point in the corrected image on a plane coordinate system and included angles between the polar meridians corresponding to each pixel point in the corrected image and an x-axis in the plane coordinate system;
the plane coordinate system is established by taking a parallel vehicle body direction as a y axis, a vertical vehicle body direction as an x axis and a point projected on a horizontal plane by the fisheye camera as an origin.
3. The image processing method according to claim 2, wherein correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficients of the glass mirror for objects at different angles, the polar meridians corresponding to the projection points of each pixel point in the corrected image on the plane coordinate system, and the angles between the polar meridians corresponding to each pixel point in the corrected image and the x-axis in the plane coordinate system includes:
correcting pixel coordinates of each pixel point in the corrected image according to the following formula;
wherein m is 00 、m 01 、m 10 、m 11 、m 20 And m 21 The parameters in the internal reference matrix of the fisheye camera are k as a constant, r represents the polar distance corresponding to the projection point of any pixel point in the image on the plane coordinate system under the plane coordinate system established by taking the parallel vehicle body direction as the y axis, the vertical vehicle body direction as the x axis and the point of the fisheye camera projected on the horizontal plane as the origin, θ represents the included angle between the polar distance corresponding to any pixel point and the x axis, M (θ) The observation magnification coefficient of the object with the polar angle theta of the glass mirror under the plane coordinate system is shown.
4. The image processing method according to claim 3, wherein the observation magnification factor is M (θ) ＝-0.6(π/2+β 0 -θ) 2 +0.001(π/2+β 0 - θ) +0.1, where β 0 And the included angle between the projection line of sight of the driver and the y axis is shown, and the projection line of sight of the driver is a connecting line formed by the projection of the eye point of the driver on the plane coordinate system and the origin.
5. The image processing method according to claim 3, wherein before correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for the object at the different angle, the method further comprises:
and establishing a plane coordinate system by taking a parallel vehicle body direction as a y axis, a vertical vehicle body direction as an x axis and a point projected on a horizontal plane by the fisheye camera as an origin.
6. The image processing method according to claim 1, wherein the distortion correction of the distorted image captured by the fisheye camera comprises:
and carrying out distortion correction on the distorted image shot by the fisheye camera by using a Zhengyou plane calibration method and a polynomial model transformation algorithm.
7. An image processing apparatus for correcting a distorted image captured by a fisheye camera on an automobile, comprising:
the distortion correcting unit is used for carrying out distortion correction on the distorted image shot by the fisheye camera to obtain corrected images with consistent amplification coefficients at all positions;
the coordinate correction unit is used for correcting pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficients of the glass mirror to objects with different angles to obtain a corrected image;
the observation magnification coefficient of the glass mirror for objects with different angles is characterized by the ratio between the size of the objects observed by drivers with different angles through the glass mirror and the size of the objects actually observed by the drivers.
8. An image processing apparatus comprising a memory, a processor and a transceiver in communication with each other in sequence, wherein the memory is adapted to store a computer program and the transceiver is adapted to receive and transmit messages, and wherein the processor is adapted to read the computer program to perform the image processing method according to any one of claims 1 to 6.
9. A computer-readable storage medium, having stored thereon instructions that, when executed on a computer, perform the image processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310294732.5A CN116433512A (en) | 2023-03-23 | 2023-03-23 | Image processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310294732.5A CN116433512A (en) | 2023-03-23 | 2023-03-23 | Image processing method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116433512A true CN116433512A (en) | 2023-07-14 |
Family
ID=87082501
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310294732.5A Pending CN116433512A (en) | 2023-03-23 | 2023-03-23 | Image processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116433512A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117011185A (en) * | 2023-08-21 | 2023-11-07 | 自行科技(武汉)有限公司 | Electronic rearview mirror CMS image correction method and system and electronic rearview mirror |
| CN117541517A (en) * | 2024-01-05 | 2024-02-09 | 深圳市欧冶半导体有限公司 | Dual-curvature imaging method, device, computer equipment and storage medium of CMS |
-
2023
- 2023-03-23 CN CN202310294732.5A patent/CN116433512A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117011185A (en) * | 2023-08-21 | 2023-11-07 | 自行科技(武汉)有限公司 | Electronic rearview mirror CMS image correction method and system and electronic rearview mirror |
| CN117011185B (en) * | 2023-08-21 | 2024-04-19 | 自行科技(武汉)有限公司 | Electronic rearview mirror CMS image correction method and system and electronic rearview mirror |
| CN117541517A (en) * | 2024-01-05 | 2024-02-09 | 深圳市欧冶半导体有限公司 | Dual-curvature imaging method, device, computer equipment and storage medium of CMS |
| CN117541517B (en) * | 2024-01-05 | 2024-03-08 | 深圳市欧冶半导体有限公司 | Dual-curvature imaging method, device, computer equipment and storage medium of CMS |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109712194B (en) | Vehicle-mounted all-round looking system, three-dimensional calibration method thereof and computer readable storage medium | |
| CN116433512A (en) | Image processing method, device and storage medium | |
| CN109741455B (en) | Vehicle-mounted stereoscopic panoramic display method, computer readable storage medium and system | |
| CN110381255B (en) | Vehicle-mounted video monitoring system and method applying 360-degree panoramic looking-around technology | |
| CN101636297B (en) | Vehicle periphery monitoring device | |
| US10007853B2 (en) | Image generation device for monitoring surroundings of vehicle | |
| EP2061234A1 (en) | Imaging apparatus | |
| CN102915525B (en) | Method for correcting barrel distortion of images for full-frame fish-eye lenses | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN112224132A (en) | Vehicle panoramic all-around obstacle early warning method | |
| CN110728638A (en) | Image distortion correction method, vehicle machine and vehicle | |
| CN112489136B (en) | Calibration method, position determination device, electronic equipment and storage medium | |
| CN113362228A (en) | Method and system for splicing panoramic images based on improved distortion correction and mark splicing | |
| EP4141621A1 (en) | Data processing method and device thereof | |
| CN111768332A (en) | Splicing method of vehicle-mounted all-around real-time 3D panoramic image and image acquisition device | |
| CN112584114A (en) | Head-up display and image correction method thereof | |
| CN110264395A (en) | A kind of the camera lens scaling method and relevant apparatus of vehicle-mounted monocular panorama system | |
| CN114897684A (en) | Vehicle image splicing method and device, computer equipment and storage medium | |
| CN105335927A (en) | Fisheye image processing method and device as well as vehicle | |
| US11377027B2 (en) | Image processing apparatus, imaging apparatus, driving assistance apparatus, mobile body, and image processing method | |
| CN116486351A (en) | Driving early warning method, device, equipment and storage medium | |
| CN117078769A (en) | Method and system for calibrating external parameters of CMS camera of electronic rearview mirror | |
| CN111626227A (en) | Method for realizing vehicle bottom perspective panoramic system based on binocular vision | |
| CN105323447A (en) | Multi-fisheye image processing method and device and vehicle | |
| CN115439551A (en) | External reference calibration method and device for vehicle-mounted panoramic camera, medium and vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |