CN115936986A - Panoramic image generation method, device, equipment and storage medium - Google Patents
Panoramic image generation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115936986A CN115936986A CN202211540712.3A CN202211540712A CN115936986A CN 115936986 A CN115936986 A CN 115936986A CN 202211540712 A CN202211540712 A CN 202211540712A CN 115936986 A CN115936986 A CN 115936986A
- Authority
- CN
- China
- Prior art keywords
- image
- vehicle
- dimensional
- panoramic image
- chassis area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Processing (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a panoramic image generation method, a panoramic image generation device, panoramic image generation equipment and a storage medium, and belongs to the technical field of image processing. The invention generates a plurality of continuous three-dimensional aerial views; acquiring vehicle information of a vehicle in a driving process; generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views; acquiring an image of the surrounding environment of the vehicle body; the method comprises the steps of fusing an image of a chassis area with an image of the environment around the vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area, constructing a more accurate image of the chassis area by using a three-dimensional aerial view, realizing the visualization of the chassis area, and enabling the chassis area and images shot by other camera areas to be perfect, seamless and undifferentiated panoramic images in a fusion mode, so that the image mapping sense of the chassis area is eliminated, and the whole panoramic image is more real.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a panoramic image generation method, apparatus, device, and storage medium.
Background
At present, in the 3D around-the-eye system development of suppliers and host manufacturers, in order to achieve a wider field of view to improve driving safety, the functions of the 3D transparent chassis are developed, but most of the existing transparent chassis functions in the market have the following disadvantages: because the 3D transparent chassis area image adopts the other four cameras to acquire pixels by adopting the image data of the previous frame, when the acquired large-angle turning is carried out, the wheel image is easily extracted into the chassis area. The contrast of the chassis area and other 3D projection views is obvious, the chassis area is displayed suddenly and has an obvious rectangular frame, the 3D chassis area has an obvious misalignment problem, and the rectangular edge of the chassis area has obvious saw teeth, so that in the generated panoramic image, the image of the chassis area has obvious and sudden chartless chartlet, and the generated panoramic image is not true enough.
The above is only for the purpose of assisting understanding of the technical solution of the present invention, and does not represent an admission that the above is the prior art.
Disclosure of Invention
The invention mainly aims to provide a panoramic image generation method, a panoramic image generation device, panoramic image generation equipment and a panoramic image generation storage medium, and aims to solve the technical problem that images of a chassis area acquired in the prior art are not accurate enough, so that the images of the chassis area and other projection views have obvious and abrupt chartlet during fusion, and the generated panoramic image is not real enough.
In order to achieve the above object, the present invention provides a panoramic image generation method, including the steps of:
generating a plurality of continuous three-dimensional aerial views;
acquiring vehicle information of a vehicle in a driving process;
generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views;
acquiring an image of the surrounding environment of the vehicle body;
and fusing the image of the chassis area and the image of the environment around the vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area.
Optionally, the generating an image of a chassis area 5 based on the vehicle information and the plurality of consecutive three-dimensional aerial views comprises:
performing photometric alignment processing on the multiple continuous three-dimensional aerial views to obtain multiple processed continuous three-dimensional aerial views;
determining a current vehicle attitude and a current position based on the vehicle information;
and filling pixels in the chassis area according to the current vehicle posture, the current position and the processed continuous three-dimensional 0-dimensional aerial views to obtain an image of the chassis area.
Optionally, the pixel filling of the chassis area according to the current vehicle attitude, the current position, and the processed plurality of consecutive three-dimensional bird's-eye views comprises:
selecting a three-dimensional aerial view of the current time from the processed multiple continuous three-dimensional aerial views based on the time corresponding to each three-dimensional aerial view;
5 processing the plurality of continuous three-dimensional images from the processed plurality of continuous three-dimensional images according to the current vehicle attitude and the current position
Selecting a three-dimensional aerial view at the previous moment from the aerial view;
determining an area covered by a vehicle chassis according to the three-dimensional aerial view at the current moment and the three-dimensional aerial view at the previous moment;
and extracting a pixel corresponding to 0 from the three-dimensional aerial view at the previous moment, wherein the pixel is covered by the vehicle chassis, and filling the chassis area by the pixel.
Optionally, the selecting a three-dimensional bird's eye view at a previous time from the processed plurality of consecutive three-dimensional bird's eye views according to the current vehicle attitude and the current position includes:
determining the position of the vehicle at the moment according to the current vehicle posture and the current position;
and selecting the three-dimensional aerial view at the previous moment from the processed multiple continuous three-dimensional aerial view 5 images according to the position of the vehicle at the previous moment.
Optionally, the generating a plurality of consecutive three-dimensional aerial views comprises:
constructing an image conversion lookup table;
acquiring two-dimensional images of the surrounding environment of the vehicle body at different moments by using cameras arranged on the periphery of the vehicle body;
and converting the two-dimensional 0 image of the environment around the vehicle body into a plurality of continuous three-dimensional aerial views based on the image conversion lookup table and a preset projection model.
Optionally, the constructing an image transformation look-up table includes:
calibrating the internal and external parameters of the cameras arranged around the vehicle body;
determining the corresponding relation between the two-dimensional images shot by the cameras and the three-dimensional coordinate points on the preset projection model according to the calibrated internal and external parameters;
and constructing an image transformation lookup table according to the corresponding relation.
Optionally, the fusing the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image includes:
fitting the image of the chassis area with a chassis area corresponding to a preset projection model in a texture mapping mode to obtain a target chassis area image;
and fusing the target chassis area image and the image of the surrounding environment of the vehicle body to obtain a panoramic image.
In order to achieve the above object, the present invention also provides a panoramic image generation apparatus including:
the generation module is used for generating a plurality of continuous three-dimensional aerial views;
the reading module is used for acquiring vehicle information of a vehicle in the running process;
a processing module for generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views;
the shooting module is used for acquiring an image of the environment around the vehicle body;
and the fusion module is used for fusing the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image, and the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area.
Further, to achieve the above object, the present invention also proposes a panoramic image generation apparatus comprising: a memory, a processor and a panoramic image generation program stored on the memory and run on the processor, the panoramic image generation program configured to implement the panoramic image generation method as described above.
Further, to achieve the above object, the present invention also proposes a storage medium having stored thereon a panoramic image generation program which, when executed by a processor, implements the panoramic image generation method as described above.
The invention generates a plurality of continuous three-dimensional aerial views; acquiring vehicle information of a vehicle in a driving process; generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views; acquiring an image of the surrounding environment of the vehicle body; the method comprises the steps of fusing an image of a chassis area with an image of an environment around a vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area, constructing an image of the chassis area which is more accurate by utilizing a three-dimensional aerial view, realizing visualization of the chassis area, and enabling the chassis area and images shot by other camera areas to be perfect, seamless and undifferentiated through a fusion mode, so that the image mapping sense of the chassis area is eliminated, and the whole panoramic image is more real.
Drawings
Fig. 1 is a schematic structural diagram of a panoramic image generation apparatus in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a panoramic image generation method according to a first embodiment of the present invention;
FIG. 3 is a schematic overall flowchart of panoramic image generation according to an embodiment of the panoramic image generation method of the present invention;
FIG. 4 is a flowchart illustrating a panoramic image generating method according to a second embodiment of the present invention;
FIG. 5 is a schematic view of a three-dimensional bird's eye view diagram in an embodiment of a panoramic image generation method according to the present invention;
FIG. 6 is a schematic view of a three-dimensional bird's eye view diagram in an embodiment of a panoramic image generation method according to the present invention;
FIG. 7 is a flowchart illustrating a panoramic image generating method according to a third embodiment of the present invention;
fig. 8 is a block diagram showing the configuration of the panoramic image generation apparatus according to the first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a panoramic image generation device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the panoramic image generation apparatus may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the panoramic image generation apparatus, and may include more or less components than those shown, or combine certain components, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and a panoramic image generation program.
In the panoramic image generation apparatus shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the panoramic image generation apparatus of the present invention may be provided in a panoramic image generation apparatus that calls a panoramic image generation program stored in the memory 1005 through the processor 1001 and executes a panoramic image generation method provided by an embodiment of the present invention.
An embodiment of the present invention provides a panoramic image generation method, and referring to fig. 2, fig. 2 is a schematic flowchart of a first embodiment of a panoramic image generation method according to the present invention.
In this embodiment, the panoramic image generation method includes the following steps:
step S10: a plurality of consecutive three-dimensional aerial views are generated.
In this embodiment, the execution subject of this embodiment may be the panoramic image generation apparatus having functions of data processing, data communication, program execution, and the like, and the panoramic image generation apparatus may be an in-vehicle terminal. Of course, other devices with similar functions may be used, and the present embodiment is not limited thereto. For convenience of explanation, the present embodiment will be described taking a panoramic image generating apparatus as an example.
It should be noted that, in the development of the 3D panoramic system by the supplier and the host manufacturer, in order to achieve a wider viewing area and improve driving safety, the functions of the 3D transparent chassis are developed, but most of the functions of the existing transparent chassis on the market have the following disadvantages: because the 3D transparent chassis area image adopts the other four cameras to acquire pixels by adopting the image data of the previous frame, when the acquired large-angle turning is carried out, the wheel image is easily extracted into the chassis area. The contrast of the chassis area and other 3D projection views is obvious, the chassis area is displayed in a projecting manner and has an obvious rectangular frame, the 3D chassis area has an obvious misalignment problem, and the rectangular edge of the chassis area has obvious saw teeth.
In order to solve the above problem, in this embodiment, an image of a chassis area of a vehicle is generated through a three-dimensional bird's eye view, and then the image of the chassis area of the vehicle is fused with images of other areas, that is, images of other areas shot by a camera on the vehicle, perfectly seamlessly and indiscriminately, so as to obtain a panoramic image.
In a specific implementation, the overall flow of the present embodiment is described with reference to fig. 3 as an example.
Referring to fig. 3, the generation of the panoramic image in the present embodiment includes three parts: a Bird's Eye View (BEV) look-up table (LUT) generation phase, a Transparent Chassis (TC) generation phase, and a fusion phase. In order to accelerate the speed of image generation BEV, in this embodiment, an LUT required for a 3D BEV view is solved according to a transformation relationship between a projection model and a BEV view, where a correspondence relationship between a two-dimensional image and the projection model is stored in the LUT, and it should be noted that a single view captured by a camera of a vehicle is a two-dimensional image, and the two-dimensional image captured by the camera of the vehicle can be converted into a three-dimensional bird's eye view according to the LUT constructed. Furthermore, before generating the LUT of the BEV, internal and external parameters of the camera are calibrated, and then the LUT of the BEV is obtained according to matrix transformation through the calibrated internal and external parameters and the BEV coordinates of the projection model.
In a specific implementation, the projection model may adopt a BEV grid, the BEV grid is provided with BEV coordinates, a correspondence between a two-dimensional image shot by a camera and the BEV coordinates on the BEV grid may be determined according to calibrated internal and external parameters of the camera, and the correspondence is stored to obtain the LUT.
As further shown in fig. 3, in this embodiment, after obtaining the 3D BEV, the obtained 3D BEV is processed in a photometric alignment manner, and then a 3D TC area image is generated in combination with a motion state of the vehicle, specifically, vehicle posture data may be obtained by an Inertial Measurement Unit (IMU) of the vehicle, and a real-time TC area image may be generated according to the BEV view generated in the previous stage and the obtained vehicle IMU posture data, that is, a pixel is extracted from an image at a time point on the BEV view and supplemented to a chassis area, and after a period of time elapses, an invisible chassis area may be filled to be visible. Before the transparent chassis is realized, the chassis area is invisible, after an image of the transparent picture area is obtained, the image of the transparent chassis needs to be attached to the chassis area in a texture mapping mode, and in order to enable the chassis area to be seamlessly and indiscriminately fused with projected images of other four cameras, an image fusion technology needs to be used for fusing the chassis area with other views, so that a final seamless and indiscriminate 3D full-view panoramic image containing the transparent chassis is obtained, wherein the image fusion technology can adopt opengl, and certainly, other modes can be selected according to actual situations for image fusion, which is not limited in this embodiment.
In a specific implementation, it is first necessary to generate a plurality of consecutive three-dimensional bird's-eye views, that is, three-dimensional bird's-eye views at a plurality of consecutive time points during the driving of the vehicle, where the three-dimensional bird's-eye view obtained at each time point is different due to different environments around the vehicle.
Step S20: vehicle information of a vehicle in the process of running is acquired.
In this embodiment, in the embodiment, an image of a chassis area of a vehicle is generated in real time, it is required to combine a motion state of the vehicle, vehicle information of the vehicle during driving may be collected by an inertial unit disposed on the vehicle, for example, the vehicle information collected by the inertial unit includes a posture or a motion state of the vehicle, and the vehicle information may also be obtained by a positioning system, for example, the vehicle information collected by the positioning system includes a position where the vehicle is located.
Step S30: generating an image of the chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views.
In a specific implementation, the motion state and the current position of the vehicle can be determined based on the vehicle information in this embodiment, then the three-dimensional bird's-eye view at the current time and the three-dimensional bird's-eye view at the previous time are screened out from a plurality of consecutive three-dimensional bird's-eye views, then the invisible area covered by the chassis of the vehicle due to the movement of the vehicle can be determined by comparing the three-dimensional bird's-eye view at the current time and the three-dimensional bird's-eye view at the previous time, finally an image of the invisible area is generated by pixel filling, and a complete image of the chassis area can be finally obtained by pixel filling for a plurality of times.
Step S40: an image of the environment surrounding the vehicle body is acquired.
In this embodiment, the panoramic image finally generated in this embodiment may display the environment around the vehicle body and the environment in the chassis area, the image of the vehicle chassis is obtained in the above steps to realize a transparent chassis, and after the image of the vehicle chassis is obtained, the image of the environment around the vehicle body needs to be further obtained in this embodiment, so that the panoramic image is a panoramic image with a full view angle and including the transparent chassis.
It should be noted that, in this embodiment, a plurality of cameras are installed around the vehicle body, and images of the environment around the vehicle body can be acquired through these cameras.
Step S50: and fusing the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area.
In a specific implementation, after obtaining the image of the chassis area and the image of the environment around the vehicle body, in this embodiment, the image of the chassis area and the image of the environment around the vehicle body are fused in an image fusion manner, so as to obtain a panoramic image including the transparent chassis.
Further, in order to enable the image of the chassis area to be seamlessly and indiscriminately fused with the projected image of another camera (that is, the image of the environment around the vehicle body), in this embodiment, the image of the chassis area is firstly attached to the chassis area corresponding to the preset projection model in a texture-attachment manner, so as to obtain an image of the target chassis area, where the preset projection model may use BEV grids, and the image of the chassis area is further attached to the chassis area in the foregoing manner, and then the image of the target chassis area is fused with the image of the environment around the vehicle body by using a fusion technology, so as to obtain a panoramic image.
The embodiment generates a plurality of continuous three-dimensional aerial views; acquiring vehicle information of a vehicle in a driving process; generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views; acquiring an image of the surrounding environment of the vehicle body; the method comprises the steps of fusing an image of a chassis area with an image of the environment around the vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area, constructing a more accurate image of the chassis area by using a three-dimensional aerial view, realizing the visualization of the chassis area, and enabling the chassis area and images shot by other camera areas to be perfect, seamless and undifferentiated panoramic images in a fusion mode, so that the image mapping sense of the chassis area is eliminated, and the whole panoramic image is more real.
Referring to fig. 4, fig. 4 is a flowchart illustrating a panoramic image generating method according to a second embodiment of the present invention.
Based on the first embodiment, in the panoramic image generation method according to this embodiment, the step S30 specifically includes:
step S301: and performing photometric alignment processing on the plurality of continuous three-dimensional aerial views to obtain a plurality of processed continuous three-dimensional aerial views.
In this embodiment, after the plurality of consecutive three-dimensional bird's-eye views are generated, the plurality of consecutive three-dimensional bird's-eye views are further aligned in this embodiment, and the plurality of consecutive three-dimensional bird's-eye views may be aligned in a photometric alignment manner. An implementation is provided for optical alignment in this example, and in particular, the image may be aligned based on the photometric consistency of all high-contrast pixels, including corners, edges and high-texture regions, in this embodiment. Of course, in this embodiment, the photometric alignment processing may be performed on a plurality of consecutive three-dimensional bird's-eye views in other manners, and an appropriate manner may be selected based on the image processing requirement, which is not limited in this embodiment.
Step S302: a current vehicle attitude and a current position are determined based on the vehicle information.
In a specific implementation, after obtaining the vehicle information, the present embodiment may determine the current vehicle attitude by the vehicle collected by the inertial unit and determine the current position where the vehicle is located according to the vehicle information collected by the positioning system.
Step S303: and filling pixels in the chassis area according to the current vehicle posture, the current position and the processed multiple continuous three-dimensional aerial views to obtain an image of the chassis area.
In specific implementation, the chassis area is invisible, and a camera mounted on a vehicle body cannot directly acquire an image of the chassis area, so that in this embodiment, the chassis area is filled in a pixel filling manner, so as to obtain a final image of the chassis area. Specifically, in the present embodiment, the chassis area may be pixel-filled based on the current vehicle attitude, the current position, and the processed multiple consecutive three-dimensional bird's-eye views.
It should be noted that the three-dimensional bird's eye view shows the environment around the vehicle, which includes the road environment in front of the vehicle, but cannot directly show the environment of the chassis area of the vehicle, however, as the vehicle moves, the environment around the vehicle shown in the three-dimensional bird's eye view at each time is different, and the included road environment in front of the vehicle is naturally different, and in this embodiment, the area covered by the chassis during the movement of the vehicle can be determined by the three-dimensional bird's eye views at different times.
In a specific implementation, the area covered by the vehicle chassis may be determined according to the three-dimensional bird's eye view image at the current time and the three-dimensional bird's eye view image at the previous time, after the area covered by the vehicle chassis is determined, in this embodiment, pixels corresponding to the area covered by the vehicle chassis are further extracted from the three-dimensional bird's eye view image at the previous time, then the chassis area is filled by the pixels, and after a period of time, an image of the chassis area may be obtained, as shown in fig. 5 and 6, assuming that T is T in fig. 5 0 A three-dimensional bird's eye view taken at all times, FIG. 6 is T 1 Time-captured three-dimensional aerial view, T 0 Is T 1 When the vehicle continues to move forward, that is, when the three-dimensional bird's-eye view at the next time T1 is shown in fig. 6, a-b becomes the area covered by the chassis of the vehicle at the time T1, so that the area covered by the chassis can be obtained by the three-dimensional bird's-eye views at different times. Wherein, the three-dimensional bird's-eye view at the current moment can be selected from the processed plurality of continuous three-dimensional bird's-eye views based on the corresponding moments of the three-dimensional bird's-eye views, the three-dimensional bird's-eye view at the previous moment can be selected from the processed plurality of continuous three-dimensional bird's-eye views according to the current vehicle posture and the current position, specifically, the position of the vehicle at the previous moment is determined according to the current vehicle posture and the current position, then the three-dimensional bird's-eye view at the previous moment is selected from the processed plurality of continuous three-dimensional bird's-eye views according to the position of the vehicle at the previous moment, for example, the current vehicle is at the A position, and the vehicle can be determined to be in the straight line according to the current vehicle posture, therefore, the position of the vehicle at the previous moment can be determined to be on the same straight line with the A position, and the three-dimensional bird's-eye view at the previous moment can be selected from the plurality of momentsAnd screening the three-dimensional aerial view at the previous moment from the subsequent three-dimensional aerial view.
Selecting a three-dimensional aerial view of the current time from the processed multiple continuous three-dimensional aerial views based on the time corresponding to each three-dimensional aerial view; determining the position of the vehicle at the moment according to the current vehicle posture and the current position; selecting a three-dimensional aerial view of the last moment from the processed multiple continuous three-dimensional aerial views according to the position of the vehicle at the last moment; determining an area covered by a vehicle chassis according to the three-dimensional aerial view at the current moment and the three-dimensional aerial view at the previous moment; and extracting pixels corresponding to the area covered by the vehicle chassis from the three-dimensional aerial view at the previous moment, filling the chassis area through the pixels, and acquiring a more real and accurate chassis area image in a pixel filling mode, thereby realizing the transparent chassis.
Referring to fig. 7, fig. 7 is a flowchart illustrating a panoramic image generating method according to a third embodiment of the present invention.
Based on the first embodiment described above, a third embodiment of a panoramic image generation method of the present invention is proposed.
In this embodiment, the step S10 specifically includes:
step S101: and constructing an image transformation lookup table.
It should be noted that, in the present embodiment, the single views captured by the camera are both two-dimensional views, and need to be converted into a three-dimensional bird's-eye view, and in order to accelerate the generation of the three-dimensional bird's-eye view, in the present embodiment, the corresponding relationship required for conversion, that is, an image conversion look-up table, is constructed in advance, and the generation of the three-dimensional bird's-eye view can be accelerated by using the image conversion look-up table.
Further, in this embodiment, an implementation manner is provided for the process of constructing the image transformation lookup table, in a specific implementation, the preset projection model may be a bird's-eye view grid, coordinates corresponding to a three-dimensional bird's-eye view are provided on the bird's-eye view grid, a correspondence between a two-dimensional image of an environment around a vehicle body captured by the camera and the coordinates corresponding to the three-dimensional bird's-eye view on the bird's-eye view grid may be determined according to internal and external parameters of the calibrated camera, and the image transformation lookup table is constructed and obtained based on the correspondence.
Step S102: the two-dimensional images of the surrounding environment of the vehicle body at different moments are obtained through the cameras arranged on the periphery of the vehicle body.
In specific implementation, the three-dimensional bird's eye view in the present embodiment is converted from a single-view image (i.e., a two-dimensional image of the environment around the vehicle body) captured by a camera, so that it is necessary to acquire two-dimensional images of the environment around the vehicle body at different times by the cameras arranged around the vehicle body.
Step S103: and converting the two-dimensional image of the environment around the vehicle body into a plurality of continuous three-dimensional aerial views based on the image conversion lookup table and a preset projection model.
In a specific implementation, the preset projection mode may employ a bird's-eye view grid, the bird's-eye view grid is provided with coordinates corresponding to the three-dimensional bird's-eye view, and the image transformation lookup table stores a corresponding relationship between the two-dimensional image of the environment around the vehicle body captured by the camera and the coordinates corresponding to the three-dimensional bird's-eye view on the bird's-eye view grid, so that each pixel on the two-dimensional image can be converted into the coordinates corresponding to the three-dimensional bird's-eye view, and the three-dimensional bird's-eye view can be finally obtained by a set of the coordinates.
In the embodiment, the internal and external parameters of the cameras arranged around the vehicle body are calibrated; determining the corresponding relation between the two-dimensional images shot by the cameras and the three-dimensional coordinate points on the preset projection model according to the calibrated internal and external parameters; constructing an image conversion lookup table according to the corresponding relation, and acquiring two-dimensional images of the surrounding environment of the vehicle body at different moments through cameras arranged around the vehicle body; the two-dimensional image of the surrounding environment of the vehicle body is converted into a plurality of continuous three-dimensional aerial views based on the image conversion lookup table and the preset projection model, and the generation efficiency of the three-dimensional aerial views is accelerated by constructing the image conversion lookup table, so that the generation efficiency of the whole panoramic image is ensured.
Furthermore, an embodiment of the present invention also proposes a storage medium having a panoramic image generation program stored thereon, which when executed by a processor implements the steps of the panoramic image generation method as described above.
Since the storage medium adopts all technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are achieved, and no further description is given here.
Referring to fig. 8, fig. 8 is a block diagram showing the configuration of the panoramic image generation apparatus according to the first embodiment of the present invention.
As shown in fig. 8, a panoramic image generation apparatus according to an embodiment of the present invention includes:
the generation module 10 is used for generating a plurality of continuous three-dimensional aerial views.
The reading module 20 is used for acquiring vehicle information of the vehicle in the running process.
A processing module 30 for generating an image of the chassis area based on the vehicle information and the plurality of consecutive three-dimensional bird's eye views.
And the shooting module 40 is used for acquiring images of the environment around the vehicle body.
And the fusion module 50 is configured to fuse the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image, where the panoramic image is used to display the environment around the vehicle body and the chassis area.
The embodiment generates a plurality of continuous three-dimensional aerial views; acquiring vehicle information of a vehicle in a driving process; generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views; acquiring an image of the environment around the vehicle body; the method comprises the steps of fusing an image of a chassis area with an image of the environment around the vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area, constructing a more accurate image of the chassis area by using a three-dimensional aerial view, realizing the visualization of the chassis area, and enabling the chassis area and images shot by other camera areas to be perfect, seamless and undifferentiated panoramic images in a fusion mode, so that the image mapping sense of the chassis area is eliminated, and the whole panoramic image is more real.
In an embodiment, the processing module 30 is further configured to perform photometric alignment processing on the multiple continuous three-dimensional bird's-eye views, so as to obtain multiple processed continuous three-dimensional bird's-eye views; determining a current vehicle attitude and a current position based on the vehicle information; and filling pixels in the chassis area according to the current vehicle posture, the current position and the processed multiple continuous three-dimensional aerial views to obtain an image of the chassis area.
In an embodiment, the processing module 30 is further configured to select a three-dimensional bird's eye view image of the current time from the processed multiple consecutive three-dimensional bird's eye views based on the time corresponding to each three-dimensional bird's eye view image; selecting a three-dimensional aerial view at a previous moment from the processed plurality of continuous three-dimensional aerial views according to the current vehicle attitude and the current position; determining an area covered by a vehicle chassis according to the three-dimensional aerial view at the current moment and the three-dimensional aerial view at the previous moment; and extracting pixels corresponding to the area covered by the vehicle chassis from the three-dimensional aerial view at the previous moment, and filling the chassis area through the pixels.
In an embodiment, the processing module 30 is further configured to determine a position of the vehicle at a time point according to the current vehicle posture and the current position; and selecting the three-dimensional aerial view at the previous moment from the processed multiple continuous three-dimensional aerial views according to the position of the vehicle at the previous moment.
In an embodiment, the generating module 10 is further configured to construct an image transformation look-up table; acquiring two-dimensional images of the surrounding environment of the vehicle body at different moments by using cameras arranged on the periphery of the vehicle body; and converting the two-dimensional image of the environment around the vehicle body into a plurality of continuous three-dimensional aerial views based on the image conversion lookup table and a preset projection model.
In an embodiment, the generating module 10 is further configured to calibrate internal and external parameters of cameras disposed around a vehicle body; determining the corresponding relation between the two-dimensional images shot by the cameras and the three-dimensional coordinate points on the preset projection model according to the calibrated internal and external parameters; and constructing an image transformation lookup table according to the corresponding relation.
In an embodiment, the fusion module 50 is further configured to attach the image of the chassis region to a chassis region corresponding to a preset projection model in a texture mapping manner, so as to obtain a target chassis region image; and fusing the target chassis area image and the image of the surrounding environment of the vehicle body to obtain a panoramic image.
It should be understood that the above is only an example, and the technical solution of the present invention is not limited in any way, and in a specific application, a person skilled in the art may set the technical solution as needed, and the present invention is not limited thereto.
It should be noted that the above-mentioned work flows are only illustrative and do not limit the scope of the present invention, and in practical applications, those skilled in the art may select some or all of them according to actual needs to implement the purpose of the solution of the present embodiment, and the present invention is not limited herein.
In addition, the technical details that are not described in detail in this embodiment may refer to the panoramic image generation method provided in any embodiment of the present invention, and are not described herein again.
Further, it is to be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present invention or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g. a Read Only Memory (ROM)/RAM, a magnetic disk, and an optical disk), and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A panoramic image generation method, characterized by comprising:
generating a plurality of continuous three-dimensional aerial views;
acquiring vehicle information of a vehicle in a driving process;
generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views;
acquiring an image of the surrounding environment of the vehicle body;
and fusing the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image, wherein the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area.
2. The panoramic image generation method of claim 1, wherein the generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional bird's eye views comprises:
performing photometric alignment processing on the multiple continuous three-dimensional aerial views to obtain multiple processed continuous three-dimensional aerial views;
determining a current vehicle attitude and a current position based on the vehicle information;
and filling pixels in the chassis area according to the current vehicle posture, the current position and the processed multiple continuous three-dimensional aerial views to obtain an image of the chassis area.
3. The panoramic image generation method of claim 2, wherein the pixel filling of the chassis area according to the current vehicle pose, the current position, and the processed plurality of consecutive three-dimensional aerial views comprises:
selecting a three-dimensional aerial view of the current time from the processed multiple continuous three-dimensional aerial views based on the time corresponding to each three-dimensional aerial view;
selecting a three-dimensional aerial view at a previous moment from the processed multiple continuous three-dimensional aerial views according to the current vehicle posture and the current position;
determining an area covered by a vehicle chassis according to the three-dimensional aerial view at the current moment and the three-dimensional aerial view at the previous moment;
and extracting pixels corresponding to the area covered by the vehicle chassis from the three-dimensional aerial view at the previous moment, and filling the chassis area through the pixels.
4. The panoramic image generation method according to claim 3, wherein the selecting a three-dimensional bird's eye view at a previous time from the processed plurality of consecutive three-dimensional bird's eye views according to the current vehicle attitude and the current position includes:
determining the position of the vehicle at the last moment according to the current vehicle posture and the current position;
and selecting the three-dimensional aerial view at the previous moment from the processed multiple continuous three-dimensional aerial views according to the position of the vehicle at the previous moment.
5. The panoramic image generation method of claim 1, wherein the generating a plurality of consecutive three-dimensional bird's eye views comprises:
constructing an image conversion lookup table;
acquiring two-dimensional images of the surrounding environment of the vehicle body at different moments by using cameras arranged on the periphery of the vehicle body;
and converting the two-dimensional image of the environment around the vehicle body into a plurality of continuous three-dimensional aerial views based on the image conversion lookup table and a preset projection model.
6. The panoramic image generation method of claim 5, wherein the constructing an image transformation look-up table comprises:
calibrating internal and external parameters of cameras arranged around a vehicle body;
determining the corresponding relation between the two-dimensional images shot by the cameras and the three-dimensional coordinate points on the preset projection model according to the calibrated internal and external parameters;
and constructing an image transformation lookup table according to the corresponding relation.
7. The panoramic image generation method according to any one of claims 1 to 6, wherein the fusing the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image includes:
fitting the image of the chassis area with a chassis area corresponding to a preset projection model in a texture mapping mode to obtain a target chassis area image;
and fusing the target chassis area image and the image of the surrounding environment of the vehicle body to obtain a panoramic image.
8. A panoramic image generation apparatus, characterized by comprising:
the generation module is used for generating a plurality of continuous three-dimensional aerial views;
the reading module is used for acquiring vehicle information of a vehicle in the running process;
a processing module for generating an image of a chassis area based on the vehicle information and the plurality of consecutive three-dimensional aerial views;
the shooting module is used for acquiring an image of the environment around the vehicle body;
and the fusion module is used for fusing the image of the chassis area with the image of the environment around the vehicle body to obtain a panoramic image, and the panoramic image is used for displaying the environment around the vehicle body and the environment of the chassis area.
9. A panoramic image generation apparatus characterized by comprising: a memory, a processor, and a panoramic image generation program stored on the memory and run on the processor, the panoramic image generation program configured to implement the panoramic image generation method of any one of claims 1 to 7.
10. A storage medium characterized in that a panoramic image generation program is stored thereon, which when executed by a processor implements the panoramic image generation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211540712.3A CN115936986A (en) | 2022-12-02 | 2022-12-02 | Panoramic image generation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211540712.3A CN115936986A (en) | 2022-12-02 | 2022-12-02 | Panoramic image generation method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115936986A true CN115936986A (en) | 2023-04-07 |
Family
ID=86700131
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211540712.3A Pending CN115936986A (en) | 2022-12-02 | 2022-12-02 | Panoramic image generation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115936986A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117789152A (en) * | 2024-02-26 | 2024-03-29 | 上海励驰半导体有限公司 | Image acquisition method and device, electronic device and storage medium |
-
2022
- 2022-12-02 CN CN202211540712.3A patent/CN115936986A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117789152A (en) * | 2024-02-26 | 2024-03-29 | 上海励驰半导体有限公司 | Image acquisition method and device, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3438919B1 (en) | Image displaying method and head-mounted display apparatus | |
| CN109829981B (en) | Three-dimensional scene presentation method, device, equipment and storage medium | |
| US20070183685A1 (en) | Image combining apparatus, image combining method and storage medium | |
| JP2005339313A (en) | Method and apparatus for presenting image | |
| JP2003216982A (en) | Device and method for providing information, storage medium, and computer program | |
| WO2017029885A1 (en) | Image generating device and image display control device | |
| US20200292319A1 (en) | Systems and methods for electronic mapping and localization within a facility | |
| CN111161398B (en) | Image generation method, device, equipment and storage medium | |
| CN112215747A (en) | Method and device for generating vehicle-mounted panoramic picture without vehicle bottom blind area and storage medium | |
| JP3245336B2 (en) | Modeling method and modeling system | |
| CN110648274A (en) | Fisheye image generation method and device | |
| WO2017113729A1 (en) | 360-degree image loading method and loading module, and mobile terminal | |
| CN115936986A (en) | Panoramic image generation method, device, equipment and storage medium | |
| WO2022166868A1 (en) | Walkthrough view generation method, apparatus and device, and storage medium | |
| US20120106868A1 (en) | Apparatus and method for image correction | |
| JP2738588B2 (en) | 3D model drawing method | |
| CN108090212B (en) | Method, device and equipment for showing interest points and storage medium | |
| CN109801351B (en) | Dynamic image generation method and processing device | |
| EP3098776B1 (en) | Image generation device and operation assistance system | |
| CN115861503A (en) | Rendering method, device and equipment of virtual object and storage medium | |
| JP2022138883A (en) | Image creation method, control method, and information processing apparatus | |
| JP2005165283A (en) | Map display device | |
| CN111028357B (en) | Soft shadow processing method and device of augmented reality equipment | |
| JP2003216977A (en) | Device and method for providing information, storage medium, and computer program | |
| CN116684561B (en) | Startup image adjusting method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |