CN116238414A - Projection method, device and equipment of intelligent car lamp and storage medium - Google Patents
Projection method, device and equipment of intelligent car lamp and storage medium Download PDFInfo
- Publication number
- CN116238414A CN116238414A CN202310172726.2A CN202310172726A CN116238414A CN 116238414 A CN116238414 A CN 116238414A CN 202310172726 A CN202310172726 A CN 202310172726A CN 116238414 A CN116238414 A CN 116238414A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- image
- projection
- coordinates
- projector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
- B60Q1/26—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
- B60R16/0231—Circuits relating to the driving or the functioning of the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q2400/00—Special features or arrangements of exterior signal lamps for vehicles
- B60Q2400/50—Projected symbol or information, e.g. onto the road or car body
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Abstract
The invention discloses a projection method, a device, equipment and a storage medium of an intelligent car lamp, wherein the method comprises the following steps: acquiring an image to be projected, and determining world coordinate system coordinates of each point on the image to be projected; converting the world coordinate system coordinates into image coordinate system coordinates according to the projection relationship between the world coordinate system and the image coordinate system; and after the image coordinate system coordinates are corrected, the projector driving the intelligent car lamp images on the projection screen according to the corrected image coordinate system coordinates. According to the projection method of the intelligent car lamp, rich graphics and symbols can be projected and displayed through the projector of the intelligent car lamp, image deformation in the projection process is effectively removed, a projection picture is perfectly fused with a road environment, trip safety is improved, and meanwhile the possibility of more interactive scenes is given.
Description
Technical Field
The invention relates to the technical field of intelligent automobiles, in particular to a projection method, a projection device, projection equipment and a storage medium of an intelligent automobile lamp.
Background
The lighting lamps are indispensable to the driver during driving as an important component of the vehicle design. Along with the development of artificial intelligence technology, the traditional automobile is also being intelligently changed, and when the automobile headlight is continuously developed in light source technology, the lighting function is also more and more trend to be intelligent, so that scene-based use experience is provided for driving, and more friendly interaction is brought to other vehicles, pedestrians and the like in the automobile environment while a driver is facilitated.
The intelligent car lamp can be a high-resolution projection headlamp (HD DLP), comprises a special projection optical component, can receive ADAS signals from a controller, and can display corresponding pictures through light and shade difference projection on the ground, a wall surface and the like. The imaging principle of the DLP projector is that cold light source emitted by an ultra-high power bulb passes through a condensing lens, light is homogenized by a light rod, the processed light passes through a color wheel, the light is divided into RGB three colors, the colors are projected on a DMD chip by the lens, and finally, the colors are reflected to be imaged on a projection screen by a projection lens. However, based on the imaging principle, distortion is generated in the projection process of the DLP projector, so that the projected picture is deformed, which is a problem to be solved in the prior art.
Disclosure of Invention
The invention provides a projection method, a projection device, projection equipment and a storage medium of an intelligent car lamp, which are used for removing image deformation in the projection process.
According to an aspect of the present invention, there is provided a projection method of an intelligent vehicle lamp, including:
acquiring an image to be projected, and determining world coordinate system coordinates of each point on the image to be projected;
converting the world coordinate system coordinates into image coordinate system coordinates according to the projection relation between the world coordinate system and the image coordinate system;
and after the image coordinate system coordinates are corrected, driving the projector of the intelligent car lamp to image on a projection screen according to the corrected image coordinate system coordinates.
Further, determining world coordinate system coordinates of points on the image to be projected includes:
receiving detection results of vehicles and nearby environments where intelligent car lamps are located, and determining a projection area of the image to be projected according to the detection results;
and matching the image to be projected with the projection area, and determining the world coordinate system coordinates of each point on the image to be projected according to the world coordinate system coordinates of the projection area.
Further, converting the world coordinate system coordinates into image coordinate system coordinates according to a projection relationship between the world coordinate system and the image coordinate system, includes:
establishing a projection transformation matrix between a world coordinate system and an image coordinate system;
and according to the projection relation between the world coordinate system and the image coordinate system, combining the projection transformation matrix, determining the corresponding relation between the image coordinate system coordinate and the world coordinate system coordinate, and obtaining the image coordinate system coordinate.
Further, the correcting process for the coordinates of the image coordinate system includes:
determining a distortion parameter and an oblique transformation matrix of the projector;
and according to the distortion parameters and the oblique transformation matrix, combining a set correction formula, and determining coordinates of the corrected image coordinate system.
Further, before obtaining the world coordinate system coordinates of each point on the image to be projected, the method further comprises:
and performing internal parameter calibration and external parameter calibration on the projector, and respectively determining an internal parameter matrix and an external parameter matrix of the projector.
Further, performing internal parameter calibration on the projector, and determining an internal parameter matrix of the projector, including:
acquiring the central pixel coordinates of a calibration image for projector calibration;
and determining an internal reference matrix of the projector according to the central pixel coordinates.
Further, performing external parameter calibration on the projector, and determining an external parameter matrix of the projector, including:
projecting the calibration image through the projector to obtain a projection image of the calibration image;
and determining an external parameter matrix of the projector according to the coordinates of the corresponding points on the projection image and the calibration image and combining the conversion relation between the world coordinate system and the image coordinate system.
According to another aspect of the present invention, there is provided a projection apparatus for an intelligent vehicle lamp, comprising:
the world coordinate system coordinate determining module is used for acquiring an image to be projected and determining world coordinate system coordinates of each point on the image to be projected;
the image coordinate system coordinate conversion module is used for converting the world coordinate system coordinate into the image coordinate system coordinate according to the projection relationship between the world coordinate system and the image coordinate system;
and the driving imaging module is used for carrying out correction processing on the image coordinate system coordinates and then driving the projector of the intelligent car lamp to image on the projection screen according to the corrected image coordinate system coordinates.
Optionally, the world coordinate system coordinate determination module is further configured to:
receiving detection results of vehicles and nearby environments where intelligent car lamps are located, and determining a projection area of the image to be projected according to the detection results;
and matching the image to be projected with the projection area, and determining the world coordinate system coordinates of each point on the image to be projected according to the world coordinate system coordinates of the projection area.
Optionally, the image coordinate system coordinate conversion module is further configured to:
establishing a projection transformation matrix between a world coordinate system and an image coordinate system;
and according to the projection relation between the world coordinate system and the image coordinate system, combining the projection transformation matrix, determining the corresponding relation between the image coordinate system coordinate and the world coordinate system coordinate, and obtaining the image coordinate system coordinate.
Optionally, the driving imaging module is further configured to:
determining a distortion parameter and an oblique transformation matrix of the projector;
and according to the distortion parameters and the oblique transformation matrix, combining a set correction formula, and determining coordinates of the corrected image coordinate system.
Optionally, the projection device of the intelligent car lamp further comprises a projector calibration module, which is used for performing internal parameter calibration and external parameter calibration on the projector, and determining an internal parameter matrix and an external parameter matrix of the projector respectively.
Optionally, the projector calibration module is further configured to:
acquiring the central pixel coordinates of a calibration image for projector calibration;
and determining an internal reference matrix of the projector according to the central pixel coordinates.
Optionally, the projector calibration module is further configured to:
projecting the calibration image through the projector to obtain a projection image of the calibration image;
and determining an external parameter matrix of the projector according to the coordinates of the corresponding points on the projection image and the calibration image and combining the conversion relation between the world coordinate system and the image coordinate system.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of projecting a smart car light according to any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute the method for projecting a smart car light according to any one of the embodiments of the present invention.
The invention discloses a projection method of an intelligent car lamp, which comprises the steps of firstly obtaining an image to be projected, determining world coordinate system coordinates of each point on the image to be projected, then converting the world coordinate system coordinates into image coordinate system coordinates according to a projection relation between the world coordinate system and the image coordinate system, finally correcting the image coordinate system coordinates, and driving a projector of the intelligent car lamp to image on a projection screen according to the corrected image coordinate system coordinates. According to the projection method of the intelligent car lamp, rich graphics and symbols can be projected and displayed through the projector of the intelligent car lamp, image deformation in the projection process is effectively removed, a projection picture is perfectly fused with a road environment, trip safety is improved, and meanwhile the possibility of more interactive scenes is given.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a projection method of an intelligent car lamp according to a first embodiment of the present invention;
fig. 2 is a hardware frame diagram of a smart car lamp according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of a projector calibration according to a first embodiment of the invention;
fig. 4 is a flowchart of a projection method of an intelligent car lamp according to a second embodiment of the present invention;
FIG. 5 is a diagram illustrating a tilt correction according to a second embodiment of the present invention;
fig. 6 is a schematic structural diagram of a projection device of an intelligent car lamp according to a third embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device for implementing the projection method of the intelligent car lamp according to the fourth embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a projection method of an intelligent vehicle lamp according to an embodiment of the present invention, where the method may be performed by a projection device of the intelligent vehicle lamp, the projection device of the intelligent vehicle lamp may be implemented in hardware and/or software, and the projection device of the intelligent vehicle lamp may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, acquiring an image to be projected, and determining world coordinate system coordinates of each point on the image to be projected.
Besides the traditional illumination function, the intelligent car lamp also has a picture projection function, and can project and display rich images or accords with the images on the ground, the wall surface and the like through the light and shade difference through a special projection optical assembly. The image to be projected is the image data received by the intelligent car lamp, and the image data is projected on the display plane through the projection optical assembly, so that a corresponding projection picture can be obtained.
Further, the world coordinate system is an absolute coordinate system of the system, and the coordinates of all points on the screen before the user coordinate system is established are the real coordinates of specific points in the real space, and the positions of the points are determined by the origin of the coordinate system. In this embodiment, the world coordinate system coordinates of each point on the image to be projected are the world coordinate system coordinates of each corresponding point on the projection screen displayed after the image to be projected is projected by the intelligent vehicle lamp. Since the position of the projection screen to be displayed on the display plane is known, the world coordinate system coordinates of the points on the projection screen can be determined.
Optionally, the smart car light includes an SOC (System on Chip), which is a systematic solution for integrating a CPU (central processing unit), a GPU (graphics processor), a Digital Signal Processor (DSP), a RAM (memory), a Modem (Modem), a navigation positioning module, a multimedia module, and the like. The SOC can analyze the data, draw and generate a display picture, specifically, the SOC can analyze and process the image through OpenCV, render a graphical interface through OpenGL, and finally output the processed lamplight image as an image to be projected. Therefore, the image to be projected can be obtained by the micro control unit (Microcontroller Unit, MCU) of the intelligent car lamp performing data interaction with the SoC through SPI (Serial Perripheral Interface, serial peripheral interface) to obtain the image to be projected output by the SoC.
Fig. 2 is a hardware frame diagram of an intelligent vehicle lamp according to an embodiment of the present invention, where the intelligent vehicle lamp includes an SOC and an MCU, the SOC may draw a graph to generate an image to be projected, and output the image to projectors of left and right vehicle lamps, and the MCU controls driving modules of the left and right vehicle lamps through a CAN (Controller Area Network ) bus and drives the projectors to image.
S120, converting the world coordinate system coordinates into image coordinate system coordinates according to the projection relation between the world coordinate system and the image coordinate system.
The image coordinate system is a coordinate system established by taking a two-dimensional photo shot by a camera as a reference and is used for specifying the position of an object in the photo.
In this embodiment, the projection optical component for projection imaging in the smart car lamp may be a DLP (Digital Light Processing ) projector. The imaging principle of the DLP projector is that a cold light source emitted by an Ultra High Power (UHP) bulb passes through a condensing lens, light is homogenized by a Rod, the processed light passes through a Color Wheel (Color Wheel), the light is divided into RGB three colors, the colors are projected on a DMD (Digital Micromirror Device, digital micromirror chip) chip through a lens, and finally reflected and imaged on a projection screen through a projection lens. The projection imaging model of the DLP projector is an inverse process of camera imaging, and in order to accurately project an image to be projected on a position corresponding to world coordinate system coordinates of each point on the image to be projected, the world coordinate system coordinates need to be converted into image coordinate system coordinates, so that the DLP projector performs projection display according to the image coordinate system coordinates.
Specifically, a specific projection conversion relation exists between the world coordinate system and the image coordinate system, and the specific projection conversion relation can be expressed by the following formula:
wherein Z is c Is a scale factor, (X) w ，Y w ，Z w ) Is a coordinate in the world coordinate system,(u, v) is the image coordinates in the corresponding image coordinate system, and M is the projective transformation matrix.
Wherein, the liquid crystal display device comprises a liquid crystal display device,
wherein, because the DLP projector can not directly acquire projection image information, calibration needs to be completed with the aid of a camera, M 1 Representing an internal reference matrix of the camera, M 2 Matrix representing camera parameters, M 1 ，M 2 Respectively, only to the camera's internal and external parameters. Wherein f is the focal length of the lens, d x And d y The lateral and longitudinal sizes, u, of the pixel dimensions of the calibration image for camera calibration, respectively 0 And v 0 And respectively representing a horizontal pixel coordinate value and a vertical pixel coordinate value of the central pixel coordinate of the calibration image, wherein R is a rotation matrix, and T is a translation matrix.
In this embodiment, after the world coordinate system coordinates are determined according to the above conversion relation, the world coordinate system coordinates can be converted into image coordinate system coordinates.
And S130, after correction processing is carried out on the image coordinate system coordinates, a projector for driving the intelligent car lamp images on a projection screen according to the corrected image coordinate system coordinates.
In the process of projection imaging of the DLP projector, because light rays are deflected when passing through the lens, and the deflection of the light rays is larger at a place far away from the center of the lens, and the lens is not completely parallel to an image plane, the projected picture is distorted compared with the original image to be projected. In addition, when the DLP is projected in a direction other than the vertical direction, the projected picture is deformed in a trapezoid shape. To overcome the lens distortion and trapezoidal deformation, correction processing is required, including distortion correction and inclination correction.
In this embodiment, the correction processing may be performed by determining correction parameters, and then converting them into corrected image coordinate system coordinates based on the obtained image coordinate system coordinates in accordance with the correction parameters in combination with a set correction formula. The distortion correction parameters include a distortion correction parameter and a tilt correction parameter.
Further, after the corrected image coordinate system coordinates are obtained through correction processing, the corrected image coordinate system coordinates can be output to the DLP display driving module, and light rays emitted by the DLP image source can be imaged on a projection plane, such as a ground or a wall surface in front of a driver.
Further, before obtaining the world coordinate system coordinates of each point on the image to be projected, the method may further include: and performing internal parameter calibration and external parameter calibration on the projector, and respectively determining an internal parameter matrix and an external parameter matrix of the projector.
According to the above steps, since the DLP projector cannot directly acquire the projected image information, calibration needs to be completed with the aid of the camera, and the calibration is to determine the internal reference matrix M of the camera 1 And an external reference matrix M of the camera 2 . The correlation between the three-dimensional geometrical position of a certain point on the surface of the space object and the corresponding point in the image can be determined through calibration,
optionally, the projector is calibrated by internal parameters, and the mode of determining the internal parameter matrix of the projector may be: acquiring the central pixel coordinates of a calibration image for projector calibration; an internal reference matrix of the projector is determined from the center pixel coordinates.
Specifically, an internal reference matrix
Wherein f is the focal length of the lens, d x And d y The lateral and longitudinal sizes, u, of the pixel dimensions of the calibration image for camera calibration, respectively 0 And v 0 And respectively representing a horizontal pixel coordinate value and a vertical pixel coordinate value of the central pixel coordinate of the calibration image. For internal reference calibration, a checkerboard calibration method is adopted, namely a set checkerboard image is used as a calibration image, and the internal reference matrix M can be determined because the image coordinates of all internal corner points on the calibration image and the corresponding spatial three-dimensional coordinates are known 1 。
Optionally, the projector is calibrated by external parameters, and the mode of determining the external parameter matrix of the projector may be: projecting a calibration image through a projector to obtain a projection image of the calibration image; and determining an external parameter matrix of the projector according to the coordinates of the corresponding points on the projection image and the calibration image and combining the conversion relation between the world coordinate system and the image coordinate system.
Specifically, for a certain point p in space, it is expressed as (X w ，Y w ，Z w ) The corresponding image coordinate system has the image coordinate of (u, v) and
when external parameter calibration is carried out, the external parameter matrix can be solved according to the world coordinate system coordinates of a plurality of points and the corresponding image coordinate system coordinates by fixing the pose of a camera, projecting a calibration image and shooting the projection image for a plurality of times
The internal reference matrix and the external reference matrix together establish the transformation from the three-dimensional point homogeneous coordinates to the two-dimensional point homogeneous coordinates.
Fig. 3 is a schematic diagram of calibration of a projector according to an embodiment of the present invention, where, as shown in the drawing, left and right lamps respectively correspond to respective DLP projectors, a checkerboard image for calibration is projected by the projector, then the image projected by the projector is shot by a camera, and calibration can be completed according to a correspondence between coordinates of points on the two images, that is, an internal reference matrix and an external reference matrix are determined.
The invention discloses a projection method of an intelligent car lamp, which comprises the steps of firstly obtaining an image to be projected, determining world coordinate system coordinates of each point on the image to be projected, then converting the world coordinate system coordinates into image coordinate system coordinates according to a projection relation between the world coordinate system and the image coordinate system, finally correcting the image coordinate system coordinates, and driving a projector of the intelligent car lamp to image on a projection screen according to the corrected image coordinate system coordinates. According to the projection method of the intelligent car lamp, rich graphics and symbols can be projected and displayed through the projector of the intelligent car lamp, image deformation in the projection process is effectively removed, a projection picture is perfectly fused with a road environment, trip safety is improved, and meanwhile the possibility of more interactive scenes is given.
Example two
Fig. 4 is a flowchart of a projection method of an intelligent vehicle lamp according to a second embodiment of the present invention, where the present embodiment is a refinement of the foregoing embodiment. As shown in fig. 4, the method includes:
s210, acquiring an image to be projected, receiving detection results of the vehicle where the intelligent car lamp is located and the nearby environment, and determining a projection area of the image to be projected according to the detection results.
The detection result may include the current position of the vehicle, vehicle condition information, surrounding environment information, such as vehicle speed information, steering information, navigation coordinate information on the vehicle, whether an obstacle exists around, and the like.
In the present embodiment, the detection information may be obtained by detecting the vehicle itself and the surrounding environment of the vehicle by the vehicle machine and an advanced driving support system (Advanced Driving Assistance System, ADAS) mounted on the vehicle. The ADAS can use various sensors (millimeter wave radar, laser radar, single/double camera and satellite navigation) installed on the vehicle to sense surrounding environment at any time in the running process of the vehicle, collect data, identify, detect and track static and dynamic objects, and combine navigation map data to perform systematic operation and analysis.
Optionally, the intelligent car lamp CAN receive the detection results of the car machine and the ADAS through a CAN (Controller Area Network ) communication mode, and determine a projection area, i.e. a specific position where a projection picture is to be displayed, for example, the ground at two meters in front of the car, or the wall surface in front of the car, etc. according to the detection results.
And S220, matching the image to be projected with the projection area, and determining the world coordinate system coordinates of each point on the image to be projected according to the world coordinate system coordinates of the projection area.
In this embodiment, after the projection area is determined, the image to be projected may be matched with the projection area, for example, a boundary line of a display screen after the projection of the image to be projected may be determined according to the boundary line of the projection area, and then the corresponding position after the projection of each point on the image to be projected may be determined according to the determined boundary line. Knowing the world coordinate system of the projection area, the world coordinate system coordinates of the points on the image to be projected can be determined.
Alternatively, the boundary line of the projection area may be determined as the boundary line of the display screen after the projection of the image to be projected, so that the display screen is displayed in proportion to the image to be projected in the projection area, and further, the world coordinate system coordinates of each point on the display screen are determined according to the world coordinate system coordinates of the projection area, and are determined as the world coordinate system coordinates of the corresponding point on the image to be projected.
S230, establishing a projection transformation matrix between the world coordinate system and the image coordinate system.
The projective transformation matrix may represent a conversion relation existing between the world coordinate system and the image coordinate system.
In this embodiment, the projective transformation matrix may be expressed as:
wherein f is the focal length of the lens, d x And d y The lateral and longitudinal sizes, u, of the pixel dimensions of the calibration image for camera calibration, respectively 0 And v 0 And respectively representing a horizontal pixel coordinate value and a vertical pixel coordinate value of the central pixel coordinate of the calibration image, wherein R is a rotation matrix, and T is a translation matrix.
S240, according to the projection relation between the world coordinate system and the image coordinate system and combining the projection transformation matrix, determining the corresponding relation between the image coordinate system coordinate and the world coordinate system coordinate, and obtaining the image coordinate system coordinate.
In this embodiment, after the projective transformation matrix is determined, the corresponding coordinates of the image coordinate system may be obtained according to the determined coordinates of the world coordinate system of the point on the image to be projected according to the projective relationship between the world coordinate system and the image coordinate system.
The correspondence between the image coordinate system coordinates and the world coordinate system coordinates can be expressed as:
wherein Z is c Is a scale factor, (X) w ，Y w ，Z w ) The coordinates in the world coordinate system are, (u, v) the image coordinates in the corresponding image coordinate system, and M is the projective transformation matrix.
S250, determining a distortion parameter and an oblique transformation matrix of the projector, and determining coordinates of an image coordinate system after correction processing according to the distortion parameter and the oblique transformation matrix and combining a set correction formula.
In this embodiment, the distortion generated by projection includes radial distortion and tangential distortion, and the distortion correction formula can be expressed as:
wherein, (x, y) is the coordinate of a point on the image to be projected under the image coordinate system, r represents the distance between the point and the origin of the coordinate system, and (x ', y') is the coordinate of the image coordinate system after the point is subjected to distortion correction. k (k) 1 、k 2 、k 3 Representing the radial distortion coefficient, p 1 、p 2 Representing the tangential distortion coefficient. The distortion parameters include radial distortion coefficients and tangential distortion coefficients, which can be determined by calibration of the projector.
The tilt correction formula can be expressed as:
wherein A represents an inclination transformation matrix, (X, Y) is the coordinates of a point on an image to be projected under an image coordinate system, (X, Y, Z) is the corresponding world coordinate system coordinates, and (X ', Y ', Z ') is the corresponding world coordinate system coordinates after inclination correction.
Further, the method comprises the steps of,
the elements in the matrix may also be determined by calibration of the projector.
Fig. 5 is a schematic diagram of tilt correction provided in an embodiment of the present invention, where, as shown in the drawing, a quadrangle ABCD and a quadrangle EFGH are projection images of a left projector and a right projector before tilt correction, and after tilt correction, a quadrangle a 'B' C 'D' and a quadrangle E 'F' G 'H' are obtained respectively.
And S260, driving the projector of the intelligent car lamp to image on a projection screen according to the coordinates of the corrected image coordinate system.
In this embodiment, after the corrected coordinates of the image coordinate system are obtained through correction processing, the corrected coordinates of the image coordinate system may be output to the DLP display driving module, and the light emitted by the DLP image source may be imaged on the projection plane, for example, in a position such as a ground or a wall in front of the driver.
The invention discloses a projection method of an intelligent car lamp, which comprises the steps of firstly obtaining an image to be projected, receiving detection results of a car where the intelligent car lamp is located and a nearby environment, determining a projection area of the image to be projected according to the detection results, then matching the image to be projected with the projection area, determining world coordinate system coordinates of each point on the image to be projected according to world coordinate system coordinates of the projection area, establishing a projection transformation matrix between the world coordinate system and the image coordinate system, combining the projection transformation matrix according to projection relations between the world coordinate system and the image coordinate system, determining a corresponding relation between the image coordinate system coordinates and the world coordinate system coordinates, obtaining image coordinate system coordinates, determining distortion parameters and an inclination transformation matrix of a projector, combining and setting a correction formula according to the distortion parameters and the inclination transformation matrix, and finally driving the projector of the intelligent car lamp to image on a projection screen according to the corrected image coordinate system coordinates. According to the projection method of the intelligent car lamp, rich graphics and symbols can be projected and displayed through the projector of the intelligent car lamp, image deformation in the projection process is effectively removed, a projection picture is perfectly fused with a road environment, trip safety is improved, and meanwhile the possibility of more interactive scenes is given.
Example III
Fig. 6 is a schematic structural diagram of a projection device of an intelligent car lamp according to a third embodiment of the present invention. As shown in fig. 6, the apparatus includes: a world coordinate system coordinate determination module 310, an image coordinate system coordinate conversion module 320, and a drive imaging module 330.
The world coordinate system coordinate determining module 310 is configured to obtain an image to be projected, and determine world coordinate system coordinates of points on the image to be projected.
The image coordinate system coordinate conversion module 320 is configured to convert world coordinate system coordinates into image coordinate system coordinates according to a projection relationship between the world coordinate system and the image coordinate system.
And the driving imaging module 330 is used for carrying out correction processing on the coordinates of the image coordinate system, and then driving the projector of the intelligent car lamp to image on the projection screen according to the corrected coordinates of the image coordinate system.
Optionally, the world coordinate system coordinate determining module 310 is further configured to:
receiving detection results of vehicles and nearby environments where intelligent car lamps are located, and determining a projection area of an image to be projected according to the detection results; matching the image to be projected with the projection area, and determining the world coordinate system coordinates of each point on the image to be projected according to the world coordinate system coordinates of the projection area.
Optionally, the image coordinate system coordinate conversion module 320 is further configured to:
establishing a projection transformation matrix between a world coordinate system and an image coordinate system; and determining the corresponding relation between the coordinates of the image coordinate system and the coordinates of the world coordinate system according to the projection relation between the world coordinate system and the image coordinate system and combining the projection transformation matrix, so as to obtain the coordinates of the image coordinate system.
Optionally, the driving imaging module 330 is further configured to:
determining distortion parameters and an oblique transformation matrix of the projector; and according to the distortion parameters and the oblique transformation matrix, setting a correction formula in a combined mode, and determining coordinates of the corrected image coordinate system.
Optionally, the projection device of the intelligent car lamp further comprises a projector calibration module 340, which is used for performing internal parameter calibration and external parameter calibration on the projector, and determining an internal parameter matrix and an external parameter matrix of the projector respectively.
Optionally, the projector calibration module 340 is further configured to:
acquiring the central pixel coordinates of a calibration image for projector calibration; an internal reference matrix of the projector is determined from the center pixel coordinates.
Optionally, the projector calibration module 340 is further configured to:
projecting a calibration image through a projector to obtain a projection image of the calibration image; and determining an external parameter matrix of the projector according to the coordinates of the corresponding points on the projection image and the calibration image and combining the conversion relation between the world coordinate system and the image coordinate system.
The projection device of the intelligent car lamp provided by the embodiment of the invention can execute the projection method of the intelligent car lamp provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 7 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 7, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the projection method of a smart car light.
In some embodiments, the method of projection of the smart car light may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the projection of the intelligent vehicle lamp described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method of projection of the smart car light in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. The projection method of the intelligent car lamp is characterized by comprising the following steps of:
acquiring an image to be projected, and determining world coordinate system coordinates of each point on the image to be projected;
converting the world coordinate system coordinates into image coordinate system coordinates according to the projection relation between the world coordinate system and the image coordinate system;
and after the image coordinate system coordinates are corrected, driving the projector of the intelligent car lamp to image on a projection screen according to the corrected image coordinate system coordinates.
2. The method of claim 1, wherein determining world coordinate system coordinates of points on the image to be projected comprises:
receiving detection results of vehicles and nearby environments where intelligent car lamps are located, and determining a projection area of the image to be projected according to the detection results;
and matching the image to be projected with the projection area, and determining the world coordinate system coordinates of each point on the image to be projected according to the world coordinate system coordinates of the projection area.
3. The method of claim 1, wherein converting the world coordinate system coordinates to image coordinate system coordinates based on a projection relationship between the world coordinate system and the image coordinate system, comprises:
establishing a projection transformation matrix between a world coordinate system and an image coordinate system;
and according to the projection relation between the world coordinate system and the image coordinate system, combining the projection transformation matrix, determining the corresponding relation between the image coordinate system coordinate and the world coordinate system coordinate, and obtaining the image coordinate system coordinate.
4. The method according to claim 1, wherein correcting the image coordinate system coordinates includes:
determining a distortion parameter and an oblique transformation matrix of the projector;
and according to the distortion parameters and the oblique transformation matrix, combining a set correction formula, and determining coordinates of the corrected image coordinate system.
5. The method of claim 1, further comprising, prior to obtaining world coordinate system coordinates of points on the image to be projected:
and performing internal parameter calibration and external parameter calibration on the projector, and respectively determining an internal parameter matrix and an external parameter matrix of the projector.
6. The method of claim 5, wherein performing an internal reference calibration on the projector, determining an internal reference matrix for the projector, comprises:
acquiring the central pixel coordinates of a calibration image for projector calibration;
and determining an internal reference matrix of the projector according to the central pixel coordinates.
7. The method of claim 6, wherein performing an outlier calibration on the projector, determining an outlier matrix for the projector, comprises:
projecting the calibration image through the projector to obtain a projection image of the calibration image;
and determining an external parameter matrix of the projector according to the coordinates of the corresponding points on the projection image and the calibration image and combining the conversion relation between the world coordinate system and the image coordinate system.
8. Projection arrangement of intelligent car light, its characterized in that includes:
the world coordinate system coordinate determining module is used for acquiring an image to be projected and determining world coordinate system coordinates of each point on the image to be projected;
the image coordinate system coordinate conversion module is used for converting the world coordinate system coordinate into the image coordinate system coordinate according to the projection relationship between the world coordinate system and the image coordinate system;
and the driving imaging module is used for carrying out correction processing on the image coordinate system coordinates and then driving the projector of the intelligent car lamp to image on the projection screen according to the corrected image coordinate system coordinates.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of projecting a smart car light of any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the method of projecting a smart car light according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310172726.2A CN116238414A (en) | 2023-02-22 | 2023-02-22 | Projection method, device and equipment of intelligent car lamp and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310172726.2A CN116238414A (en) | 2023-02-22 | 2023-02-22 | Projection method, device and equipment of intelligent car lamp and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116238414A true CN116238414A (en) | 2023-06-09 |
Family
ID=86625774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310172726.2A Pending CN116238414A (en) | 2023-02-22 | 2023-02-22 | Projection method, device and equipment of intelligent car lamp and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116238414A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117549825A (en) * | 2024-01-12 | 2024-02-13 | 深圳市欧冶半导体有限公司 | Calibration method and device for car lamp control angle, computer equipment and storage medium |
-
2023
- 2023-02-22 CN CN202310172726.2A patent/CN116238414A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117549825A (en) * | 2024-01-12 | 2024-02-13 | 深圳市欧冶半导体有限公司 | Calibration method and device for car lamp control angle, computer equipment and storage medium |
| CN117549825B (en) * | 2024-01-12 | 2024-04-05 | 深圳市欧冶半导体有限公司 | Calibration method and device for car lamp control angle, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11948462B2 (en) | Image generating apparatus, image generating method, and recording medium | |
| CN110336987B (en) | Projector distortion correction method and device and projector | |
| JP5108605B2 (en) | Driving support system and vehicle | |
| JP2009129001A (en) | Operation support system, vehicle, and method for estimating three-dimensional object area | |
| US20180124378A1 (en) | Enhanced depth map images for mobile devices | |
| CN112651881B (en) | Image synthesizing method, apparatus, device, storage medium, and program product | |
| CN116238414A (en) | Projection method, device and equipment of intelligent car lamp and storage medium | |
| JP2010257357A (en) | Image processing apparatus, semiconductor data processing apparatus, and data processing system | |
| WO2023155581A1 (en) | Image detection method and apparatus | |
| WO2022166868A1 (en) | Walkthrough view generation method, apparatus and device, and storage medium | |
| CN114125411A (en) | Projection equipment correction method and device, storage medium and projection equipment | |
| CN112116655A (en) | Method and device for determining position information of image of target object | |
| US20220385867A1 (en) | Electronic apparatus and method for controlling thereof | |
| JP2009077022A (en) | Driving support system and vehicle | |
| CN114401388A (en) | Projection method, projection device, storage medium and projection equipment | |
| CN111161148B (en) | Panoramic image generation method, device, equipment and storage medium | |
| JP2016072691A (en) | Image processing system, control method of the same, and program | |
| KR20220162595A (en) | Electronic apparatus and control method thereof | |
| CN115431764B (en) | AR scale display method and device, electronic equipment and storage medium | |
| CN113888611B (en) | Method and device for determining image depth and storage medium | |
| CN113870365B (en) | Camera calibration method, device, equipment and storage medium | |
| CN117413511A (en) | Electronic device and control method thereof | |
| WO2021253189A1 (en) | Electric device, scanning method of controlling electric device, and computer readable storage medium | |
| CN114666558B (en) | Method and device for detecting definition of projection picture, storage medium and projection equipment | |
| CN116684569A (en) | Virtual three-dimensional object imaging method, device, camera and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |