CN114567758A - Automobile 360-degree panoramic video processing system and method - Google Patents

Automobile 360-degree panoramic video processing system and method Download PDF

Info

Publication number
CN114567758A
CN114567758A CN202210230400.6A CN202210230400A CN114567758A CN 114567758 A CN114567758 A CN 114567758A CN 202210230400 A CN202210230400 A CN 202210230400A CN 114567758 A CN114567758 A CN 114567758A
Authority
CN
China
Prior art keywords
automobile
video
camera
data
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210230400.6A
Other languages
Chinese (zh)
Inventor
高宏稳
陈群飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Chaoyue Intelligent Electronics Co ltd
Original Assignee
Shenzhen Chaoyue Intelligent Electronics Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Chaoyue Intelligent Electronics Co ltd filed Critical Shenzhen Chaoyue Intelligent Electronics Co ltd
Priority to CN202210230400.6A priority Critical patent/CN114567758A/en
Publication of CN114567758A publication Critical patent/CN114567758A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/363—Image reproducers using image projection screens
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00—Diagnosis, testing or measuring for television systems or their details
    • H04N17/002—Diagnosis, testing or measuring for television systems or their details for television cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The invention provides a system and a method for processing a 360-degree panoramic video of an automobile, wherein the system comprises the following steps: the panoramic video acquisition module is used for acquiring videos of the environment around the automobile close to the body and generating a first video set; the panoramic video processing module is used for generating a three-dimensional simulation scene according to the video in the first video set and acquiring information influencing the safe driving of the automobile by using the three-dimensional simulation scene; and the panoramic safety prompt module is used for providing safety prompt according to the information influencing the safe driving of the automobile. The method and the device can help the driver to see the surrounding environment conditions of the automobile nearby from a proper visual angle and a proper visual field by acquiring and processing the 360-degree panoramic video, and improve the driving safety.

Description

Automobile 360-degree panoramic video processing system and method
Technical Field
The invention relates to the technical field of intelligent traffic, in particular to a 360-degree panoramic video processing system and method for an automobile.
Background
The 360-degree automobile panoramic video is used for providing 360-degree video image information in a driving environment for an automobile driver; in the driving process of the automobile, the parking of the automobile and the crowded urban traffic condition of the automobile, a driver can better judge the road environment around the automobile through the 360-degree panoramic video, so that the collision with surrounding pedestrians and the vehicle is avoided, and the driving process is safer and more convenient; the existing 360-degree panoramic video processing of the automobile also has the problems of insufficient acquisition efficiency, relatively backward video processing method, insufficient timeliness for providing safety prompts and the like; accordingly, there is a need for an improved automotive 360 degree panoramic video processing system and method.
Disclosure of Invention
The invention provides a system and a method for processing a 360-degree panoramic video of an automobile, which can help a driver to see the surrounding situation of the automobile from a proper visual angle and a proper visual field by acquiring and processing the 360-degree panoramic video, and improve the driving safety.
The invention provides an automobile 360-degree panoramic video processing system, which comprises:
the panoramic video acquisition module is used for acquiring videos of the near-body surrounding environment of the automobile and generating a first video set;
the panoramic video processing module is used for generating a three-dimensional simulation scene according to the video in the first video set and acquiring information influencing the safe driving of the automobile by using the three-dimensional simulation scene;
and the panoramic safety prompt module is used for providing safety prompt according to the information influencing the safe driving of the automobile.
Further, the panoramic safety prompt module comprises a main display screen and an auxiliary display screen, wherein the main display screen is embedded in the central console in the automobile and is used for displaying a 360-degree panoramic video aerial view around the vehicle; the auxiliary display screen is arranged right in front of the sight line of the driver and used for displaying the near-body and peripheral aerial view of the automobile within the range of 20 degrees left and right by taking the visual angle direction of the driver as the center;
the near-body surrounding aerial view of the automobile is obtained by performing projection mapping on a near-body 360-degree panoramic video of the automobile based on an image re-projection program and in combination with the head movement angle information of a driver;
when the environment of the automobile is an open space environment, displaying a bird's-eye view around the automobile close to the automobile only through the auxiliary display screen; when the environment of the automobile is a narrow space environment, the primary display screen is opened to display the aerial view of 360-degree panoramic video around the automobile near the automobile body while the secondary display screen displays the aerial view around the automobile near the automobile body.
Furthermore, the panoramic video acquisition module acquires videos by using a plurality of 180-degree ultra-wide-angle fisheye cameras; the ultra-wide angle fisheye camera is arranged at the front, the back, the left and the right of the vehicle body according to a preset angle, and the preset angle is adjusted and set according to the area shot by the fisheye camera;
the panoramic video acquisition module comprises a video primary selection unit, a video inspection unit and a video set generation unit;
the video primary selection unit is used for periodically acquiring videos of the surrounding environment of the close body of the automobile shot by the camera according to a preset period to obtain a plurality of video sets;
the video inspection unit is used for inspecting the videos in the plurality of video sets according to a preset video inspection rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value, or the image boundary in the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video;
the video set generating unit is used for generating a video set; and in the same period, taking the video reserved by the video inspection unit as a quality qualified video, and converging the quality qualified video to generate a first video set.
Further, the panoramic video processing module comprises a three-dimensional simulation scene construction unit; the three-dimensional simulation scene construction unit comprises a camera calibration subunit, a model parameter acquisition subunit, a projection model construction subunit and a three-dimensional simulation scene generation subunit;
the camera calibration subunit is used for calibrating the internal parameters and the external parameters of the camera; calculating to obtain a first texture mapping table and a second texture mapping table according to the internal parameters and the external parameters of the camera;
the model parameter acquiring subunit is used for calculating the distance between the camera in the first video set image and the main planes of the environments at the left side and the right side of the automobile; matching the distance with the first texture mapping table to obtain a first projection model parameter; matching the distance with the second texture mapping table to obtain a second projection model parameter;
the projection model construction subunit is used for establishing a first projection model according to the first projection model parameter; establishing a second projection model according to the second projection model parameters;
the three-dimensional panorama generating subunit is used for generating a first three-dimensional simulation scene according to the first projection model, wherein the first three-dimensional simulation scene corresponds to an automobile in an open space environment; and generating a second three-dimensional simulation scene according to the second projection model, wherein the second three-dimensional simulation scene corresponds to an automobile in a narrow space environment.
Further, the model parameter obtaining subunit includes a space environment determining subunit, configured to determine that an environment in which the automobile is located is an open space or a narrow space; the method comprises the following specific steps:
performing optical flow tracking and matching on videos of the environments on the left side and the right side of the automobile, determining feature points on a main plane of a side-looking obstacle according to the change rule of the optical flow, and estimating relative depth values of the feature points according to epipolar geometry and triangulation;
transforming the side-view image of the rear-view camera to a top view, and calculating to obtain the inter-frame motion pose of the camera:
obtaining a scale factor according to a calibration result of the camera;
calculating the distance between the camera and the main planes of the environments at the left side and the right side of the automobile according to the relative depth value, the inter-frame motion pose and the scale factor;
judging whether the environment of the automobile is an open space or a narrow space according to the distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
A360-degree panoramic video processing method for an automobile comprises the following steps:
s1, collecting videos of the environment around the automobile close to the automobile, and generating a first video set;
s2, generating a three-dimensional simulation scene according to the video in the first video set, and observing the three-dimensional simulation scene to obtain information influencing the safe driving of the automobile;
and S3, providing safety prompt according to the information influencing the safe driving of the automobile.
Further, S1 includes
S101, arranging two adjacent radars on an automobile, selecting a certain moment, and respectively acquiring data of a first target at a first fixed position in the environment around the automobile by using the two radars to obtain first data and second data; analyzing the overlapping content of the first data and the second data, taking the overlapping content of the first data as third data, taking the overlapping content of the second data as fourth data, and preliminarily judging that the two radars work normally if the third data and the fourth data both correspond to the first target; performing data acquisition and analysis again after the two radars are shut down and restarted, and if the two radars are still judged to work normally, determining that the two radars work normally;
s102, selecting the same time as that in the step S101, and acquiring data of the first target located at the first fixed position in the environment around the automobile to be detected by using the camera to be detected to obtain fifth data; judging whether the corresponding targets in the fifth data and the first data or the second data are the first targets, if so, judging that the camera to be detected works normally; if not, performing correction operation on the camera to be detected;
s103, executing a correction operation on the camera, wherein the correction operation comprises the following steps: the method comprises the following steps of shutting down, resetting and restarting a camera, or adjusting the angle of a camera lens to a preset normal working angle, or adjusting the focal length of the camera lens to a preset normal working focal length; after the correction operation is finished, the operation is transferred to the step S102 to continue to be checked until the camera works normally, and the operation of detecting and correcting the performance of the camera is finished;
s104, regularly acquiring videos of the surrounding environment of the automobile close to the body, which are shot by a camera, according to a preset period to obtain a plurality of video sets;
s105, checking videos in the plurality of video sets according to a preset video checking rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value or the area range boundary corresponding to the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video;
and S106, in the same period, taking the video reserved by the video inspection unit as a quality qualified video, and converging the quality qualified video to generate a first video set.
Further, S2 includes constructing a three-dimensional simulation scene, and the specific steps include:
s201, calibrating internal parameters and external parameters of a camera; calculating to obtain a first texture mapping table and a second texture mapping table according to the internal parameters and the external parameters of the camera;
s202, calculating the distance between a camera in the first video set image and the main planes of the environments on the left side and the right side of the automobile; matching the distance with the first texture mapping table to obtain a first projection model parameter; matching the distance with the second texture mapping table to obtain a second projection model parameter;
s203, establishing a first projection model according to the first projection model parameters; establishing a second projection model according to the second projection model parameters;
s204, generating a first three-dimensional simulation scene according to the first projection model, wherein the first three-dimensional simulation scene corresponds to an automobile in an open space environment; and generating a second three-dimensional simulation scene according to the second projection model, wherein the second three-dimensional simulation scene corresponds to an automobile in a narrow space environment.
Further, S202 includes determining that the environment in which the vehicle is located is an open space or a narrow space; the method comprises the following specific steps:
s2021, carrying out optical flow tracking and matching on videos of the environment on the left side and the right side of the automobile, determining feature points on a main plane of the side-looking obstacle according to a change rule of the optical flow, and estimating relative depth values of the feature points according to epipolar geometry and triangulation;
s2022, transforming the side-view image of the rear-view camera to a top view, and calculating to obtain the inter-frame motion pose of the camera:
s2023, obtaining a scale factor according to a calibration result of the camera;
s2024, calculating to obtain the distance between the camera and the main planes of the left and right environments of the automobile according to the relative depth value, the inter-frame motion pose and the scale factor;
s2025, judging that the environment where the automobile is located is a wide space or a narrow space according to the distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
Further, the step of providing a safety prompt in S3 includes the specific steps of:
s301, acquiring first distance data between a camera and main planes of environments on the left side and the right side of an automobile, second distance data between the camera and the main planes of the environments on the front side and the rear side of the automobile and third distance data between a plane of the top of the automobile and the main planes of the environments outside the automobile in real time, and storing the first distance data and the second distance data into a preset database;
s302, constructing a collision risk analysis model based on an artificial neural network according to historical first interval data, historical second interval data and historical third interval data in the database; the collision risk analysis model inputs data as interval data and outputs data as a collision risk value;
s303, according to a collision risk analysis model, collision risk values of the first interval data, the second interval data and the third interval data are respectively determined, and a first collision risk value, a second collision risk value and a third collision risk value are obtained; sending out corresponding safety prompts according to the analysis results of the first collision risk value, the second collision risk value and the third collision risk value;
judging whether the first collision risk value is greater than or equal to a preset first collision risk threshold value or not, and if so, sending a first safety prompt to an automobile driver;
judging whether the second collision risk value is greater than or equal to a preset second collision risk threshold value or not, and if so, sending a second safety prompt to the automobile driver;
judging whether the third collision risk value is greater than or equal to a preset third collision risk threshold value, and if so, sending a third safety prompt to the automobile driver;
and S304, the automobile driver takes corresponding risk avoiding driving operation according to the first safety prompt, the second safety prompt or the third safety prompt.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a block diagram of an automotive 360 degree panoramic video processing system according to the present invention;
FIG. 2 is a flow chart of a method for processing a 360-degree panoramic video of an automobile according to the present invention;
FIG. 3 is a flowchart of a method for providing a security prompt of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
The invention provides a 360-degree panoramic video processing system of an automobile, which comprises the following components as shown in figure 1:
the panoramic video acquisition module is used for acquiring videos of the environment around the automobile close to the body and generating a first video set;
the panoramic video processing module is used for generating a three-dimensional simulation scene according to the video in the first video set and acquiring information influencing the safe driving of the automobile by using the three-dimensional simulation scene;
and the panoramic safety prompt module is used for providing safety prompt according to the information influencing the safe driving of the automobile.
The working principle of the technical scheme is as follows: the 360-degree panoramic video of the automobile can help a driver to better observe the surrounding environment of the automobile nearby, acquire traffic information in the environment where the automobile is located, and provide help on the visual field for the driver to drive the automobile, so that the safe driving of the automobile is guaranteed. The system comprises a panoramic video acquisition module, a panoramic video acquisition module and a panoramic video acquisition module, wherein the panoramic video acquisition module is used for acquiring videos of the environment around the automobile close to the body and generating a first video set; the panoramic video processing module is used for generating a three-dimensional simulation scene according to the video in the first video set and acquiring information influencing the safe driving of the automobile by using the three-dimensional simulation scene; and the panoramic safety prompt module is used for providing safety prompt according to the information influencing the safe driving of the automobile.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the driver can be helped to see the environment conditions around the automobile from a proper visual angle and a proper visual field by acquiring and processing the 360-degree panoramic video, and the driving safety is improved.
In one embodiment, the panoramic safety prompting module comprises a main display screen and a secondary display screen, wherein the main display screen is embedded in a central console in the automobile and is used for displaying a 360-degree panoramic video aerial view around the near body of the automobile; the auxiliary display screen is arranged right in front of the sight line of the driver and used for displaying the near-body and peripheral aerial view of the automobile within the range of 20 degrees left and right by taking the visual angle direction of the driver as the center;
the near-body surrounding aerial view of the automobile is obtained by performing projection mapping on a near-body 360-degree panoramic video of the automobile based on an image re-projection program and in combination with the head movement angle information of a driver;
when the environment of the automobile is an open space environment, displaying a bird's-eye view around the automobile close to the automobile only through the auxiliary display screen; when the environment of the automobile is a narrow space environment, the primary display screen is opened to display the aerial view of 360-degree panoramic video around the automobile near the automobile body while the secondary display screen displays the aerial view around the automobile near the automobile body.
The working principle of the technical scheme is as follows: considering the difference of the environment where the automobile runs, different display modes are necessary to be set; when the automobile passes through a narrow road and a road with obstacles, and is backed and climbed to a slope, the environment of other parts around the automobile body cannot be seen at the same time, and the panorama around the automobile body needs to be displayed through a main display screen; and only through the main display screen display, under the open environment condition of car, can not provide the observation visual angle of geometric proportion according to the observation custom of human eye, build real external environment for the driver, it is necessary only to open vice display screen work. The specific composition structure and implementation mode of the implementation are as follows: the panoramic safety prompt module comprises a main display screen and an auxiliary display screen, wherein the main display screen is embedded in a central console in the automobile and is used for displaying a 360-degree panoramic video aerial view around the automobile close to the body; the auxiliary display screen is arranged right in front of the sight line of the driver and used for displaying the near-body and peripheral aerial view of the automobile within the range of 20 degrees left and right by taking the visual angle direction of the driver as the center;
the near-body surrounding aerial view of the automobile is obtained by performing projection mapping on a near-body 360-degree panoramic video of the automobile based on an image re-projection program and in combination with the head movement angle information of a driver;
when the environment of the automobile is an open space environment, displaying a bird's-eye view around the automobile close to the automobile only through the auxiliary display screen; when the environment of the automobile is a narrow space environment, the primary display screen is opened to display the aerial view of 360-degree panoramic video around the automobile near the automobile body while the secondary display screen displays the aerial view around the automobile near the automobile body.
The beneficial effects of the above technical scheme are: adopt the scheme that this embodiment provided, utilize to set up main display screen and vice display screen, can go the difference of environment in place according to the car, show car close-in panoramic video all around, improved the pertinence of observing car close-in environment all around.
In one embodiment, the panoramic video acquisition module acquires videos by using a plurality of 180-degree ultra-wide-angle fisheye cameras; the ultra-wide angle fisheye camera is arranged at the front, the back, the left and the right of the vehicle body according to a preset angle, and the preset angle is adjusted and set according to the area shot by the fisheye camera;
the panoramic video acquisition module comprises a video primary selection unit, a video inspection unit and a video set generation unit;
the video primary selection unit is used for periodically acquiring videos of the surrounding environment of the close body of the automobile shot by the camera according to a preset period to obtain a plurality of video sets;
the video inspection unit is used for inspecting the videos in the plurality of video sets according to a preset video inspection rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value, or the image boundary in the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video;
the video set generating unit is used for generating a video set; and in the same period, taking the video reserved by the video inspection unit as a quality qualified video, and converging the quality qualified video to generate a first video set.
The working principle of the technical scheme is as follows: the video is formed by combining a series of continuous images, so the video image splicing technology can use the principle of the image splicing technology for reference; in order to ensure the quality of the obtained video images and the image splicing effect, the quality control inspection needs to be carried out on the video collected by the fisheye camera, when the video images to be spliced all accord with the preset quality qualified condition, the obtained video is reserved, the reserved video is integrated, and the video set meeting the requirements is obtained.
The method comprises the steps of firstly, regularly acquiring videos of the surrounding environment of the close body of the automobile shot by a camera according to a preset period to obtain a plurality of video sets; then, the videos in the plurality of video sets are inspected according to a preset video inspection rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value, or the image boundary in the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value and the boundary of an image in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video; and finally, in the same period, taking the video reserved by the video inspection unit as a quality-qualified video, and converging the quality-qualified video to generate a first video set.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the quality control inspection is carried out on the video collected by the fisheye camera, so that the quality of the obtained video image can be ensured, and the image splicing effect is ensured.
In one embodiment, the panoramic video processing module comprises a three-dimensional simulation scene construction unit; the three-dimensional simulation scene construction unit comprises a camera calibration subunit, a model parameter acquisition subunit, a projection model construction subunit and a three-dimensional simulation scene generation subunit;
the camera calibration subunit is used for calibrating the internal parameters and the external parameters of the camera; calculating to obtain a first texture mapping table and a second texture mapping table according to the internal parameters and the external parameters of the camera;
the model parameter acquiring subunit is used for calculating the distance between the camera in the first video set image and the main planes of the environments at the left side and the right side of the automobile; matching the distance with the first texture mapping table to obtain a first projection model parameter; matching the distance with the second texture mapping table to obtain a second projection model parameter;
the projection model construction subunit is used for establishing a first projection model according to the first projection model parameter; establishing a second projection model according to the second projection model parameters;
the three-dimensional panorama generating subunit is configured to generate a first three-dimensional simulation scene according to the first projection model, where the first three-dimensional simulation scene corresponds to an environment where an automobile is in an open space; and generating a second three-dimensional simulation scene according to the second projection model, wherein the second three-dimensional simulation scene corresponds to an automobile in a narrow space environment.
The working principle of the technical scheme is as follows: in order to better meet the panoramic view observation requirements of automobiles in different environments, it is necessary to establish corresponding three-dimensional simulation scenes for the different environments. In this embodiment, a first texture mapping table and a second texture mapping table are obtained by calculation according to an internal parameter and an external parameter of a camera calibrated in advance; the method comprises the steps of calculating the distance between a camera in a video image and main planes of the environment on the left side and the right side of an automobile according to different environments by analyzing actual environment information of the video image, obtaining corresponding projection model parameters through a texture mapping table, constructing a first projection model and a second projection model according to the projection model parameters, and completing construction of a three-dimensional simulation scene for an open space environment and a three-dimensional simulation scene for a narrow space environment according to the projection models.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the three-dimensional simulation panorama matched with the camera is constructed according to the different distances between the camera and the main planes of the environments at the left side and the right side of the automobile in the video image, the three-dimensional simulation panorama can be switched in a self-adaptive manner according to the different environments of the automobile, and a driver can conveniently observe the panoramic video of the environment around the automobile in a targeted manner.
In one embodiment, the model parameter obtaining subunit includes a space environment determining subunit, configured to determine that an environment where the automobile is located is an open space or a narrow space; the method specifically comprises the following steps:
performing optical flow tracking and matching on videos of the environments on the left side and the right side of the automobile, determining feature points on a main plane of a side-looking obstacle according to the change rule of the optical flow, and estimating relative depth values of the feature points according to epipolar geometry and triangulation;
transforming the side-view image of the rear-view camera to a top view, and calculating to obtain the inter-frame motion pose of the camera:
obtaining a scale factor according to a calibration result of the camera;
calculating the distance between the camera and the main planes of the environments at the left side and the right side of the automobile according to the relative depth value, the inter-frame motion pose and the scale factor;
judging whether the environment of the automobile is an open space or a narrow space according to the distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
The working principle of the technical scheme is as follows: in order to calculate the distance between the camera and the main planes of the left and right environments of the automobile, the video needs to be tracked to obtain feature points, and then matching is carried out, so that the relative motion of the camera is estimated; the optical flow can detect small motion changes of objects through a continuous image sequence, acquire image frames of videos on two sides, perform feature point tracking matching, and judge feature points on main planes of side-looking nearest obstacles (planes of objects on two sides, such as vehicle vertical planes of parking lots, green belts on driving roads and building planes nearest to roadsides) according to tracking results. In the feature point selection, firstly, an image processing interested area with larger feature point optical flow values and concentrated distribution on the same main plane is selected, the optical flow values of the feature points in the area are calculated, an optical flow value distribution histogram is calculated, a plurality of groups with larger horizontal component values and smaller vertical component values and larger quantity of optical flows are used as feature points to be selected, and then a group of feature points with pixel coordinate point distribution span meeting a preset threshold value is selected from the feature points to be selected and used as the main plane feature points.
For corresponding feature points located on the principal plane, relative depth values are estimated from epipolar geometry and triangulation. Epipolar geometry is used for describing the geometric relationship between two images, and only needs the internal reference and the relative posture of a camera; epipolar geometry is commonly used in the aspects of camera matrix recovery, pose estimation, three-dimensional reconstruction and the like; calculating an essential matrix and a basic matrix according to epipolar geometry, recovering a camera matrix through the essential matrix and decomposing to obtain an inter-frame motion pose; according to the corresponding relation between the size of the top view and the actual distance, the scale factor between the pixel distance of the top view and the actual distance is determined by the actual width of the top view area and the width of the top view; the distance between the camera and the main planes of the environment at the left side and the right side of the automobile can be calculated through the rotation matrix and the translational motion vector value in the interframe motion pose, and the method specifically comprises the following steps:
Figure BDA0003540290810000131
in the above formula, d is the distance between the camera and the main planes of the left and right environments of the automobile, and alpha1Is the normalized coordinate, alpha, of the feature point corresponding to the previous frame image2The normalized coordinates of the feature points corresponding to the next frame of image; p is a rotation matrix in the inter-frame motion pose of the camera from the previous frame image to the next frame image, n is a translation vector in the inter-frame motion pose of the camera from the previous frame image to the next frame image, and deltarThe actual width of the top view area of the video image is a preset value; δ is the width of the top view of the image, which is a preset value;
comparing the distance between the camera and the main planes of the environments on the left side and the right side of the automobile with a preset distance threshold value, and judging whether the environment where the automobile is located is a wide space or a narrow space; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the distance between the camera and the main planes of the environments at the left side and the right side of the automobile is calculated, the environment where the automobile is located is judged to be a wide space or a narrow space according to the distance, and the judgment accuracy can be ensured.
A method for processing a 360-degree panoramic video of an automobile, as shown in fig. 2, includes:
s1, collecting videos of the environment around the automobile close to the automobile, and generating a first video set;
s2, generating a three-dimensional simulation scene according to the video in the first video set, and observing the three-dimensional simulation scene to obtain information influencing the safe driving of the automobile;
and S3, providing safety prompt according to the information influencing the safe driving of the automobile.
The working principle of the technical scheme is as follows: the 360-degree panoramic video of the automobile can help a driver to better observe the environment around the automobile, acquire traffic information in the environment where the automobile is located, and provide help on the visual field for the driver to drive the automobile, so that the automobile is safely driven. The method comprises the steps of collecting videos of the environment around the automobile close to the body to generate a first video set; generating a three-dimensional simulation scene according to the video in the first video set, and acquiring information influencing the safe driving of the automobile by using the three-dimensional simulation scene; and providing a safety prompt according to the information influencing the safe driving of the automobile.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the driver can be helped to see the environment conditions around the automobile from a proper visual angle and a proper visual field by acquiring and processing the 360-degree panoramic video, and the driving safety is improved.
In one embodiment, S1 includes:
s101, arranging two adjacent radars on an automobile, selecting a certain moment, and respectively acquiring data of a first target at a first fixed position in the environment around the automobile by using the two radars to obtain first data and second data; analyzing the overlapping content of the first data and the second data, taking the overlapping content of the first data as third data, taking the overlapping content of the second data as fourth data, and preliminarily judging that the two radars work normally if the third data and the fourth data both correspond to the first target; performing data acquisition and analysis again after the two radars are shut down and restarted, and if the two radars are still judged to work normally, determining that the two radars work normally;
s102, selecting the same time as that in the step S101, and acquiring data of the first target located at the first fixed position in the environment around the automobile to be detected by using the camera to be detected to obtain fifth data; judging whether the corresponding targets in the fifth data and the first data or the second data are the first targets, if so, judging that the camera to be detected works normally; if not, performing correction operation on the camera to be detected;
s103, executing a correction operation on the camera, wherein the correction operation comprises the following steps: carrying out shutdown reset restart on the camera, or adjusting the angle of a camera lens to a preset normal working angle, or adjusting the focal length of the camera lens to a preset normal working focal length; after the correction operation is finished, the operation is transferred to the step S102 to continue to be checked until the camera works normally, and the operation of detecting and correcting the performance of the camera is finished;
s104, regularly acquiring videos of the surrounding environment of the close body of the automobile shot by a camera according to a preset period to obtain a plurality of video sets;
s105, checking videos in the plurality of video sets according to a preset video checking rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value or the area range boundary corresponding to the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video;
and S106, in the same period, taking the video reserved by the video inspection unit as a quality qualified video, and converging the quality qualified video to generate a first video set.
The working principle of the technical scheme is as follows: in order to ensure the quality of the obtained video images and the image splicing effect, the quality control inspection needs to be carried out on the video collected by the fisheye camera, when the video images to be spliced all accord with the preset quality qualified condition, the obtained video is reserved, the reserved video is integrated, and the video set meeting the requirements is obtained. Meanwhile, the working performance of the camera needs to be checked, and the mechanical problems are corrected and adjusted in time so as to ensure the normal and stable operation of the camera.
The specific steps of the embodiment include:
s101, arranging two adjacent radars on an automobile, selecting a certain moment, and respectively acquiring data of a first target at a first fixed position in the environment around the automobile by using the two radars to obtain first data and second data; analyzing the overlapping content of the first data and the second data, taking the overlapping content of the first data as third data, taking the overlapping content of the second data as fourth data, and preliminarily judging that the two radars work normally if the third data and the fourth data both correspond to the first target; performing data acquisition and analysis again after the two radars are shut down and restarted, and if the two radars are still judged to work normally, determining that the two radars work normally;
s102, selecting the same time as that in the step S101, and acquiring data of the first target located at the first fixed position in the environment around the automobile to be detected by using the camera to be detected to obtain fifth data; judging whether the corresponding targets in the fifth data and the first data or the second data are the first targets, if so, judging that the camera to be detected works normally; if not, performing correction operation on the camera to be detected;
s103, executing a correction operation on the camera, wherein the correction operation comprises the following steps: carrying out shutdown reset restart on the camera, or adjusting the angle of a camera lens to a preset normal working angle, or adjusting the focal length of the camera lens to a preset normal working focal length; after the correction operation is finished, the operation is transferred to the step S102 to continue to be checked until the camera works normally, and the operation of detecting and correcting the performance of the camera is finished;
s104, regularly acquiring videos of the surrounding environment of the automobile close to the body, which are shot by a camera, according to a preset period to obtain a plurality of video sets;
s105, checking videos in the plurality of video sets according to a preset video checking rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value or the area range boundary corresponding to the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video;
and S106, in the same period, taking the video reserved by the video inspection unit as a quality qualified video, and converging the quality qualified video to generate a first video set.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the normal working performance of the camera can be ensured by checking and correcting the working performance of the camera, so that the quality of the shot video is ensured; through carrying out quality control inspection on the video collected by the fisheye camera, the quality of the obtained video image can be ensured, and the image splicing effect is ensured.
In one embodiment, S2 includes constructing a three-dimensional simulation scene, specifically:
s201, calibrating internal parameters and external parameters of a camera; calculating to obtain a first texture mapping table and a second texture mapping table according to the internal parameters and the external parameters of the camera;
s202, calculating the distance between a camera in the first video set image and the main planes of the environments on the left side and the right side of the automobile; matching the distance with the first texture mapping table to obtain a first projection model parameter; matching the distance with the second texture mapping table to obtain a second projection model parameter;
s203, establishing a first projection model according to the first projection model parameters; establishing a second projection model according to the second projection model parameters;
s204, generating a first three-dimensional simulation scene according to the first projection model, wherein the first three-dimensional simulation scene corresponds to an automobile in an open space environment; and generating a second three-dimensional simulation scene according to the second projection model, wherein the second three-dimensional simulation scene corresponds to an automobile in a narrow space environment.
The working principle of the technical scheme is as follows: in order to better meet the panoramic view observation requirements of automobiles in different environments, corresponding three-dimensional simulation scenes need to be established for the different environments. In this embodiment, a first texture mapping table and a second texture mapping table are obtained by calculation according to an internal parameter and an external parameter of a camera calibrated in advance; the method comprises the steps of calculating the distance between a camera in a video image and main planes of the environment on the left side and the right side of an automobile according to different environments by analyzing actual environment information of the video image, obtaining corresponding projection model parameters through a texture mapping table, constructing a first projection model and a second projection model according to the projection model parameters, and completing construction of a three-dimensional simulation scene for an open space environment and a three-dimensional simulation scene for a narrow space environment according to the projection models.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the three-dimensional simulation panorama matched with the camera is constructed according to the different distances between the camera and the main planes of the environments at the left side and the right side of the automobile in the video image, the three-dimensional simulation panorama can be switched in a self-adaptive manner according to the different environments of the automobile, and a driver can conveniently observe the panoramic video of the environment around the automobile in a targeted manner.
In one embodiment, S202 includes determining whether the environment in which the vehicle is located is an open space or a narrow space; the method comprises the following specific steps:
s2021, carrying out optical flow tracking and matching on videos of the environment on the left side and the right side of the automobile, determining feature points on a main plane of the side-looking obstacle according to a change rule of the optical flow, and estimating relative depth values of the feature points according to epipolar geometry and triangulation;
s2022, transforming the side-view image of the rear-view camera to a top view, and calculating to obtain the inter-frame motion pose of the camera:
s2023, obtaining a scale factor according to a calibration result of the camera;
s2024, calculating to obtain the distance between the camera and the main planes of the left and right environments of the automobile according to the relative depth value, the inter-frame motion pose and the scale factor;
s2025, judging that the environment where the automobile is located is a wide space or a narrow space according to the distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
The working principle of the technical scheme is as follows: in order to calculate the distance between the camera and the main planes of the environment on the left side and the right side of the automobile, the video is tracked to obtain feature points, and then matching is carried out, so that the relative motion of the camera is estimated; the optical flow can detect small motion changes of objects through a continuous image sequence, acquire image frames of videos on two sides, perform feature point tracking matching, and judge feature points on main planes of side-looking nearest obstacles (planes of objects on two sides, such as vehicle vertical planes of parking lots, green belts on driving roads and building planes nearest to roadsides) according to tracking results. In the feature point selection, firstly, an image processing interested area with larger feature point optical flow values and concentrated distribution on the same main plane is selected, the optical flow values of the feature points in the area are calculated, an optical flow value distribution histogram is calculated, a plurality of groups with larger horizontal component values and smaller vertical component values and larger quantity of optical flows are used as feature points to be selected, and then a group with the distribution span of pixel coordinate points meeting a preset threshold value is taken out from the feature points to be selected and used as the main plane feature points.
For corresponding feature points located on the principal plane, relative depth values are estimated from epipolar geometry and triangulation. Epipolar geometry is used for describing the geometric relationship between two images, and only needs the internal reference and the relative posture of a camera; epipolar geometry is commonly used in the aspects of camera matrix recovery, pose estimation, three-dimensional reconstruction and the like; calculating an essential matrix and a basic matrix according to the epipolar geometry, recovering a camera matrix through the essential matrix and decomposing to obtain an inter-frame motion pose; according to the corresponding relation between the size of the top view and the actual distance, the scale factor between the pixel distance of the top view and the actual distance is determined by the actual width of the top view area and the width of the top view; the distance between the camera and the main planes of the environment on the left side and the right side of the automobile can be calculated through the rotation matrix and the translational motion vector value in the interframe motion pose; judging whether the environment of the automobile is an open space or a narrow space according to the calculated distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than the preset distance threshold value, judging that the automobile is in the open space environment.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the distance between the camera and the main planes of the environments at the left side and the right side of the automobile is calculated, the environment where the automobile is located is judged to be a wide space or a narrow space according to the distance, and the judgment accuracy can be ensured.
In one embodiment, as shown in fig. 3, the step of providing the safety prompt in S3 includes the following specific steps:
s301, acquiring first distance data between a camera and main planes of environments on the left side and the right side of an automobile, second distance data between the camera and the main planes of the environments on the front side and the rear side of the automobile and third distance data between a plane of the top of the automobile and the main planes of the environments outside the automobile in real time, and storing the first distance data and the second distance data into a preset database;
s302, constructing a collision risk analysis model based on an artificial neural network according to historical first interval data, historical second interval data and historical third interval data in the database; the collision risk analysis model inputs data as interval data and outputs data as a collision risk value;
s303, according to a collision risk analysis model, collision risk values of the first interval data, the second interval data and the third interval data are respectively determined, and a first collision risk value, a second collision risk value and a third collision risk value are obtained; sending out corresponding safety prompts according to the analysis results of the first collision risk value, the second collision risk value and the third collision risk value;
judging whether the first collision risk value is greater than or equal to a preset first collision risk threshold value or not, and if so, sending a first safety prompt to an automobile driver;
judging whether the second collision risk value is greater than or equal to a preset second collision risk threshold value or not, and if so, sending a second safety prompt to the automobile driver;
judging whether the third collision risk value is greater than or equal to a preset third collision risk threshold value, and if so, sending a third safety prompt to the automobile driver;
and S304, the automobile driver takes corresponding risk avoiding driving operation according to the first safety prompt, the second safety prompt or the third safety prompt.
The working principle of the technical scheme is as follows: in order to better provide safety prompts for drivers, it is necessary to distinguish different safety risks and provide corresponding prompts, so as to help the drivers to take corresponding risk avoidance driving operations to ensure that the automobiles are safely driven. The method comprises the following specific steps:
acquiring first distance data between a camera and main planes of the environment on the left side and the right side of the automobile, second distance data between the camera and the main planes of the environment behind the front part of the automobile and third distance data between the plane of the top of the automobile and the main plane of the environment outside the automobile in real time, and storing the first distance data and the second distance data into a preset database;
constructing a collision risk analysis model based on an artificial neural network according to historical first interval data, historical second interval data and historical third interval data in the database; the collision risk analysis model inputs data as interval data and outputs data as a collision risk value;
according to a collision risk analysis model, collision risk values of the first interval data, the second interval data and the third interval data are respectively determined, and a first collision risk value, a second collision risk value and a third collision risk value are obtained; sending out corresponding safety prompts according to the analysis results of the first collision risk value, the second collision risk value and the third collision risk value;
judging whether the first collision risk value is greater than or equal to a preset first collision risk threshold value or not, and if so, sending a first safety prompt to an automobile driver;
judging whether the second collision risk value is greater than or equal to a preset second collision risk threshold value or not, and if so, sending a second safety prompt to the automobile driver;
judging whether the third collision risk value is greater than or equal to a preset third collision risk threshold value, and if so, sending a third safety prompt to the automobile driver;
and the automobile driver takes corresponding risk avoiding driving operation according to the first safety prompt, the second safety prompt or the third safety prompt.
The beneficial effects of the above technical scheme are: by adopting the scheme provided by the embodiment, the possible collision risk is analyzed by analyzing the distance between the automobile and the main plane of the environment where the automobile is located according to the difference of the distances, so that safety prompts can be provided in a targeted manner, a driver is helped to avoid the collision risk in time, and safe driving is ensured.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (10)

1. An automotive 360-degree panoramic video processing system, comprising:
the panoramic video acquisition module is used for acquiring videos of the environment around the automobile close to the body and generating a first video set;
the panoramic video processing module is used for generating a three-dimensional simulation scene according to the video in the first video set and acquiring information influencing the safe driving of the automobile by using the three-dimensional simulation scene;
and the panoramic safety prompt module is used for providing safety prompt according to the information influencing the safe driving of the automobile.
2. The system of claim 1, wherein the panoramic safety prompting module comprises a main display screen and a secondary display screen, the main display screen is embedded in a center console in the automobile and is used for displaying a bird's-eye view of the 360-degree panoramic video around the automobile; the auxiliary display screen is arranged right in front of the sight line of the driver and used for displaying the near-body and peripheral aerial view of the automobile within the range of 20 degrees left and right by taking the visual angle direction of the driver as the center;
the near-body surrounding aerial view of the automobile is obtained by performing projection mapping on a near-body 360-degree panoramic video of the automobile based on an image re-projection program and in combination with the head movement angle information of a driver;
when the environment of the automobile is an open space environment, displaying a bird's-eye view around the automobile close to the automobile only through the auxiliary display screen; when the environment of the automobile is a narrow space environment, the primary display screen is opened to display the aerial view of 360-degree panoramic video around the automobile near the automobile body while the secondary display screen displays the aerial view around the automobile near the automobile body.
3. The 360-degree panoramic video processing system for the automobile according to claim 1, wherein the panoramic video collecting module is used for collecting videos by using a plurality of 180-degree ultra-wide angle fisheye cameras; the ultra-wide angle fisheye camera is arranged at the front, the back, the left and the right of the vehicle body according to a preset angle, and the preset angle is adjusted and set according to the area shot by the fisheye camera;
the panoramic video acquisition module comprises a video primary selection unit, a video inspection unit and a video set generation unit;
the video primary selection unit is used for periodically acquiring videos of the surrounding environment of the close body of the automobile shot by the camera according to a preset period to obtain a plurality of video sets;
the video inspection unit is used for inspecting the videos in the plurality of video sets according to a preset video inspection rule; the video inspection rule comprises: evaluating the definition of an image in the video and judging the boundary of the image in the video; when the image definition in the video is smaller than a preset image definition threshold value, or the image boundary in the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is overlapped with a preset image boundary, judging that the video meets the requirement, and reserving the video;
the video set generating unit is used for generating a video set; and in the same period, taking the video reserved by the video inspection unit as a quality qualified video, and converging the quality qualified video to generate a first video set.
4. The 360-degree panoramic video processing system for automobiles according to claim 1, wherein the panoramic video processing module comprises a three-dimensional simulation scene construction unit; the three-dimensional simulation scene construction unit comprises a camera calibration subunit, a model parameter acquisition subunit, a projection model construction subunit and a three-dimensional simulation scene generation subunit;
the camera calibration subunit is used for calibrating the internal parameters and the external parameters of the camera; calculating to obtain a first texture mapping table and a second texture mapping table according to the internal parameters and the external parameters of the camera;
the model parameter acquiring subunit is used for calculating the distance between the camera in the first video set image and the main planes of the environments at the left side and the right side of the automobile; matching the distance with the first texture mapping table to obtain a first projection model parameter; matching the distance with the second texture mapping table to obtain a second projection model parameter;
the projection model construction subunit is used for establishing a first projection model according to the first projection model parameter; establishing a second projection model according to the second projection model parameters;
the three-dimensional panorama generating subunit is configured to generate a first three-dimensional simulation scene according to the first projection model, where the first three-dimensional simulation scene corresponds to an environment where an automobile is in an open space; and generating a second three-dimensional simulation scene according to the second projection model, wherein the second three-dimensional simulation scene corresponds to an automobile in a narrow space environment.
5. The system of claim 3, wherein the model parameter obtaining subunit comprises a space environment determining subunit, configured to determine whether the environment of the vehicle is an open space or a narrow space; the method specifically comprises the following steps:
performing optical flow tracking and matching on videos of the environments on the left side and the right side of the automobile, determining feature points on a main plane of a side-looking obstacle according to the change rule of the optical flow, and estimating relative depth values of the feature points according to epipolar geometry and triangulation;
transforming the side-view image of the rear-view camera to a top view, and calculating to obtain the inter-frame motion pose of the camera:
obtaining a scale factor according to a calibration result of the camera;
calculating the distance between the camera and the main planes of the environments at the left side and the right side of the automobile according to the relative depth value, the inter-frame motion pose and the scale factor;
judging whether the environment of the automobile is an open space or a narrow space according to the distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
6. A360-degree panoramic video processing method for an automobile is characterized by comprising the following steps:
s1, collecting videos of the environment around the automobile close to the automobile, and generating a first video set;
s2, generating a three-dimensional simulation scene according to the video in the first video set, and observing the three-dimensional simulation scene to obtain information influencing the safe driving of the automobile;
and S3, providing safety prompt according to the information influencing the safe driving of the automobile.
7. The method according to claim 6, wherein S1 includes
S101, arranging two adjacent radars on an automobile, selecting a certain moment, and respectively acquiring data of a first target at a first fixed position in the environment around the automobile by using the two radars to obtain first data and second data; analyzing the overlapping content of the first data and the second data, taking the overlapping content of the first data as third data, taking the overlapping content of the second data as fourth data, and preliminarily judging that the two radars work normally if the third data and the fourth data both correspond to the first target; performing data acquisition and analysis again after the two radars are shut down and restarted, and if the two radars are still judged to work normally, determining that the two radars work normally;
s102, selecting the same time as that in the step S101, and acquiring data of the first target located at the first fixed position in the environment around the automobile to be detected by using the camera to be detected to obtain fifth data; judging whether the corresponding targets in the fifth data and the first data or the second data are the first targets, if so, judging that the camera to be detected works normally; if not, performing correction operation on the camera to be detected;
s103, executing a correction operation on the camera, wherein the correction operation comprises the following steps: carrying out shutdown reset restart on the camera, or adjusting the angle of a camera lens to a preset normal working angle, or adjusting the focal length of the camera lens to a preset normal working focal length; after the correction operation is finished, the operation is transferred to the step S102 to continue to be checked until the camera works normally, and the operation of detecting and correcting the performance of the camera is finished;
s104, regularly acquiring videos of the surrounding environment of the close body of the automobile shot by a camera according to a preset period to obtain a plurality of video sets;
s105, checking videos in the plurality of video sets according to a preset video checking rule; the video inspection rule comprises: evaluating the image definition in the video and judging the area range corresponding to the video; when the image definition in the video is smaller than a preset image definition threshold value or the area range boundary corresponding to the video is larger than or smaller than a preset image boundary, judging that the video does not meet the requirement, and discarding the video; when the image definition in the video is greater than or equal to a preset image definition threshold value or the image boundary in the video is superposed with a preset image boundary, judging that the video is qualified in quality and reserving the video;
and S106, in the same period, taking the videos reserved by the video inspection unit as quality-qualified videos, and converging the quality-qualified videos to generate a first video set.
8. The method for processing the 360-degree panoramic video of the automobile according to claim 6, wherein S2 includes constructing a three-dimensional simulation scene, specifically:
s201, calibrating internal parameters and external parameters of a camera; calculating to obtain a first texture mapping table and a second texture mapping table according to the internal parameters and the external parameters of the camera;
s202, calculating the distance between a camera in the first video set image and the main planes of the environments on the left side and the right side of the automobile; matching the distance with the first texture mapping table to obtain a first projection model parameter; matching the distance with the second texture mapping table to obtain a second projection model parameter;
s203, establishing a first projection model according to the first projection model parameters; establishing a second projection model according to the second projection model parameters;
s204, generating a first three-dimensional simulation scene according to the first projection model, wherein the first three-dimensional simulation scene corresponds to an automobile in an open space environment; and generating a second three-dimensional simulation scene according to the second projection model, wherein the second three-dimensional simulation scene corresponds to an automobile in a narrow space environment.
9. The method according to claim 8, wherein S202 comprises determining whether the environment of the vehicle is open or narrow, and comprises the following steps:
s2021, carrying out optical flow tracking and matching on videos of the environment on the left side and the right side of the automobile, determining feature points on a main plane of the side-looking obstacle according to a change rule of the optical flow, and estimating relative depth values of the feature points according to epipolar geometry and triangulation;
s2022, transforming the side-view image of the rear-view camera to a top view, and calculating to obtain the inter-frame motion pose of the camera:
s2023, obtaining a scale factor according to a calibration result of the camera;
s2024, calculating to obtain the distance between the camera and the main planes of the left and right environments of the automobile according to the relative depth value, the inter-frame motion pose and the scale factor;
s2025, judging that the environment where the automobile is located is a wide space or a narrow space according to the distance and the preset distance threshold value; when the distance is smaller than a preset distance threshold value, judging that the automobile is in a narrow space environment; and when the distance is larger than a preset distance threshold value, judging that the automobile is in an open space environment.
10. The method for processing the 360-degree panoramic video of the automobile according to claim 6, wherein S3 includes providing a safety prompt, and the specific steps are as follows:
s301, acquiring first distance data between a camera and main planes of environments on the left side and the right side of an automobile, second distance data between the camera and the main planes of the environments on the front side and the rear side of the automobile and third distance data between a plane of the top of the automobile and the main planes of the environments outside the automobile in real time, and storing the first distance data and the second distance data into a preset database;
s302, constructing a collision risk analysis model based on an artificial neural network according to historical first interval data, historical second interval data and historical third interval data in the database; the collision risk analysis model inputs data as interval data and outputs data as a collision risk value;
s303, according to a collision risk analysis model, collision risk values of the first interval data, the second interval data and the third interval data are respectively determined, and a first collision risk value, a second collision risk value and a third collision risk value are obtained; sending out corresponding safety prompts according to the analysis results of the first collision risk value, the second collision risk value and the third collision risk value;
judging whether the first collision risk value is greater than or equal to a preset first collision risk threshold value or not, and if so, sending a first safety prompt to an automobile driver;
judging whether the second collision risk value is greater than or equal to a preset second collision risk threshold value or not, and if so, sending a second safety prompt to the automobile driver;
judging whether the third collision risk value is greater than or equal to a preset third collision risk threshold value, and if so, sending a third safety prompt to the automobile driver;
and S304, the automobile driver takes corresponding risk avoiding driving operation according to the first safety prompt, the second safety prompt or the third safety prompt.
CN202210230400.6A 2022-03-10 2022-03-10 Automobile 360-degree panoramic video processing system and method Pending CN114567758A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210230400.6A CN114567758A (en) 2022-03-10 2022-03-10 Automobile 360-degree panoramic video processing system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210230400.6A CN114567758A (en) 2022-03-10 2022-03-10 Automobile 360-degree panoramic video processing system and method

Publications (1)

Publication Number Publication Date
CN114567758A true CN114567758A (en) 2022-05-31

Family

ID=81717024

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210230400.6A Pending CN114567758A (en) 2022-03-10 2022-03-10 Automobile 360-degree panoramic video processing system and method

Country Status (1)

Country Link
CN (1) CN114567758A (en)

Similar Documents

Publication Publication Date Title
US9858639B2 (en) Imaging surface modeling for camera modeling and virtual view synthesis
Oliveira et al. Multimodal inverse perspective mapping
CN105844624B (en) Combined optimization method and device in dynamic calibration system, dynamic calibration system
US8872920B2 (en) Camera calibration apparatus
JP5588812B2 (en) Image processing apparatus and imaging apparatus using the same
Li et al. Easy calibration of a blind-spot-free fisheye camera system using a scene of a parking space
CN101404122B (en) Driving support device, driving support method, and computer program
JP5455124B2 (en) Camera posture parameter estimation device
CN110910453B (en) Vehicle pose estimation method and system based on non-overlapping view field multi-camera system
US20150042799A1 (en) Object highlighting and sensing in vehicle image display systems
CN103802725B (en) A kind of new vehicle carried driving assistant images generation method
CN109087251B (en) Vehicle-mounted panoramic image display method and system
US20090167856A1 (en) Reverse sensing system displaying overhead vehicle image
KR20090103165A (en) Monocular Motion Stereo-Based Free Parking Space Detection Apparatus and Method
JP2010136289A (en) Device and method for supporting drive
CN110719411B (en) Panoramic all-around view image generation method of vehicle and related equipment
JP6552448B2 (en) Vehicle position detection device, vehicle position detection method, and computer program for vehicle position detection
CN109883433B (en) Vehicle positioning method in structured environment based on 360-degree panoramic view
CN111640062A (en) Automatic splicing method for vehicle-mounted all-around images
CN111141311A (en) Evaluation method and system of high-precision map positioning module
CN110780287A (en) Distance measurement method and distance measurement system based on monocular camera
CN114765972A (en) Display method, computer program, controller and vehicle for representing a model of the surroundings of a vehicle
CN112215214A (en) Method and system for adjusting camera offset of intelligent vehicle-mounted terminal
CN114567758A (en) Automobile 360-degree panoramic video processing system and method
CN113362232A (en) Vehicle panoramic all-around image generation method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination