CN112990117B - Installation data processing method and device based on intelligent driving system - Google Patents
Installation data processing method and device based on intelligent driving system Download PDFInfo
- Publication number
- CN112990117B CN112990117B CN202110429222.5A CN202110429222A CN112990117B CN 112990117 B CN112990117 B CN 112990117B CN 202110429222 A CN202110429222 A CN 202110429222A CN 112990117 B CN112990117 B CN 112990117B
- Authority
- CN
- China
- Prior art keywords
- image
- installation
- preset
- angle deviation
- camera sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention provides an installation data processing method and device based on an intelligent driving system, wherein the method comprises the following steps: acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor; judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point; and if the camera sensor is judged not to meet the preset installation compliance condition, sending out early warning information of installation non-compliance. The method can realize efficient and accurate detection judgment and early warning on the installation compliance of the camera sensor.
Description
Technical Field
The invention relates to the technical field of vehicle-mounted camera sensor installation management, in particular to an installation data processing method and device based on an intelligent driving system.
Background
With the development of electric vehicles, the safety and functionality of the vehicle driving become more and more important. An Advanced Driver assistance System (ADAS System for short, also called intelligent driving System) is installed on a vehicle, and implements an auxiliary driving technique such as a front vehicle collision warning, a lane line deviation warning and the like by using various vehicle-mounted sensors and combining with various data algorithms, so as to enhance driving safety and enhance user experience. Among the plurality of vehicle-mounted sensors, a vehicle-mounted camera sensor (which may be a vehicle-mounted camera) installed in the middle of the front windshield is considered as the most important sensor in the ADAS system because the acquired data is the basic data for the operation of the plurality of sensors. In order to make the accuracy of the whole early warning result of the ADAS system higher, the requirement on the installation compliance degree of the camera sensor is higher and higher. Further, since the mounting position or angle of the image sensor may be changed due to non-standardization of self-mounting by a user, vibration of the camera due to long-term running of the vehicle, or disassembly and reassembly of the image sensor, it is important to detect whether or not the mounting position of the image sensor is in compliance.
At present, technical methods aiming at the installation condition management of the camera sensor are few, and even if the technical methods are available, the installation state of the vehicle-mounted camera is simply and broadly determined according to parameters such as the rotation angle of the vehicle-mounted camera and the slope of the horizon line, and the detection accuracy is poor.
Disclosure of Invention
The invention provides an installation data processing method and device based on an intelligent driving system, which are used for overcoming the defects that in the prior art, the installation compliance detection method of a camera sensor is less and the detection accuracy of a common method is poor, and realizing efficient and accurate detection judgment and early warning on the installation compliance of the camera sensor.
The invention provides an installation data processing method based on an intelligent driving system, which comprises the following steps:
acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor;
judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point;
and if the camera sensor is judged not to meet the preset installation compliance condition, sending out early warning information of installation non-compliance.
According to the installation data processing method based on the intelligent driving system, whether the situation meets the installation compliance condition is judged, the situation comprises pitch angle deviation judgment, deviation angle deviation judgment and roll angle deviation judgment, and the pitch angle deviation judgment, the deviation angle deviation judgment and the roll angle deviation judgment correspond to the judgment of whether the image transverse horizontal line is overlapped with the physical horizon line or not, the judgment of whether the image longitudinal central axis is overlapped with the perpendicular bisector of the physical vehicle head or not and the judgment of whether the image central point is overlapped with the extension vanishing point or not.
According to the installation data processing method based on the intelligent driving system, aiming at the pitch angle deviation judgment and/or the deviation angle deviation judgment, the method for judging whether the camera sensor meets the preset installation compliance condition or not according to the content of the driving image comprises the following steps:
judging whether the total number of the target detection frames in the driving image exceeds a first preset threshold value or not;
when the total number of the target detection frames exceeds the first preset threshold value, detecting whether the upper edge line and the lower edge line of each target detection frame are in a first preset fluctuation range relative to the physical horizon;
and determining whether pitch angle deviation and/or offset angle deviation exists according to the proportion of the number of the target detection frames in the first preset fluctuation range relative to the physical horizon to the total number.
According to the installation data processing method based on the intelligent driving system, the target detection frame is determined according to a target detection model which takes other small motor vehicles in the driving image as detection targets and is trained by combining a YOLO open source algorithm and/or an SSD open source algorithm.
According to the installation data processing method based on the intelligent driving system, aiming at the pitch angle deviation judgment and/or the deviation angle deviation judgment and/or the roll angle deviation judgment, whether the camera sensor meets the preset installation compliance condition or not is judged according to the content of the driving image, and the method further comprises the following steps:
judging whether the running speed of the current vehicle exceeds a second preset threshold value or not;
when the running vehicle speed exceeds the second preset threshold value, determining a vanishing point position by a vanishing point position judging method, and judging whether the vanishing point position is in a second preset fluctuation range around the image center point;
it is determined whether a pitch angle deviation and/or a yaw angle deviation and/or a roll angle deviation is present.
According to the installation data processing method based on the intelligent driving system, the method for determining the vanishing point position through the vanishing point position judgment method comprises the following steps:
setting an interested area for the driving image according to the target view range;
detecting at least two lane lines from the region of interest of the driving image by Canny edge detection and/or hough line detection;
and performing operation processing on the detected at least two lane lines through a Randac algorithm to determine a vanishing point position.
According to the installation data processing method based on the intelligent driving system, whether the camera sensor meets the preset installation compliance condition or not is judged regularly according to the content of the driving image according to the preset time interval.
The invention provides an installation data processing device based on an intelligent driving system, which comprises:
the acquisition module is used for acquiring a running image of the current vehicle acquired by the vehicle-mounted camera sensor;
a judging module, configured to judge whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, where the preset installation compliance condition at least includes: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point;
and the early warning module is used for sending out early warning information of installation failure if the camera sensor is judged not to meet the preset installation failure condition.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein when the processor executes the computer program, all or part of the steps of the installation data processing method based on the intelligent driving system are realized.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements all or part of the steps of the installation data processing method based on an intelligent driving system according to any one of the above.
The invention provides an installation data processing method and device based on an intelligent driving system, and the method provides a novel camera sensor installation compliance detection method and can realize efficient and accurate detection judgment and early warning on the installation compliance of a camera sensor.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of an installation data processing method based on an intelligent driving system provided by the invention;
FIG. 2 is a second flowchart of the installation data processing method based on the intelligent driving system provided by the present invention;
FIG. 3 is a third flowchart of an installation data processing method based on an intelligent driving system according to the present invention;
FIG. 4 is a schematic structural diagram of an installation data processing device based on an intelligent driving system provided by the invention;
fig. 5 is a schematic structural diagram of an electronic device provided in the present invention.
Reference numerals:
410: an acquisition module; 420: a judgment module; 430: an early warning module;
510: a processor; 520: a communication interface; 530: a memory; 540 communication bus.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following describes an installation data processing method and device based on an intelligent driving system according to the present invention with reference to fig. 1 to 5.
The invention provides an installation data processing method based on an intelligent driving system, and fig. 1 is one of flow charts of the installation data processing method based on the intelligent driving system, as shown in fig. 1, the method comprises the following steps:
100. and acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor.
The vehicle-mounted camera sensor can be any device capable of collecting driving images in the driving process, such as a vehicle-mounted camera, a vehicle-mounted micro video recorder and a vehicle data recorder, and the device is generally installed in front of a vehicle head or at a suitable position inside and outside a front windshield of a vehicle, for example, the device is fixedly installed at a lower position in the middle of one side, close to the inside of the vehicle, of the front windshield, or is fixedly installed at positions, such as the side edge of a reflector in a common front windshield, together with a common vehicle data recorder.
And acquiring a running image which is acquired by a vehicle-mounted camera sensor in real time and is used for the current vehicle to run forwards along the running route in a straight line.
200. Judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal image horizontal line is coincided with the physical horizon, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point.
And analyzing specific contents in the driving image, wherein the specific contents comprise the position, the coordinate, the edge line and the like of each reference object in the image, and the specific reference objects can be the head of the current vehicle, vehicles beside the driving image, two or more lane lines on the driving road surface, and even can be the reference objects such as a street and overpass.
The preset installation compliance condition includes at least the following three conditions, in other words, at least the sum of the following three conditions, and none of the three conditions is as follows: the horizontal image horizontal line is overlapped with the physical horizon, the longitudinal image central axis is overlapped with the perpendicular bisector of the physical vehicle head, and the image central point is overlapped with the extended vanishing point. However, if one condition is not met, the vehicle-mounted camera sensor is installed in a non-compliant manner, so that the accuracy of the driving related data recorded and collected by the camera sensor as the basic data of the intelligent driving system cannot be met.
300. And if the camera sensor is judged not to meet the preset installation compliance condition, sending out early warning information of installation non-compliance.
If the camera sensor is judged to meet the preset installation compliance condition, the camera sensor records the collected driving related data to be used as the basic data of the intelligent driving system; and if the camera sensor is judged not to meet the preset installation compliance condition, automatically sending out early warning information of installation non-compliance according to the judgment result.
The installation data processing method based on the intelligent driving system can perform detailed judgment and detection on the preset installation compliance condition and perform effective early warning under the condition of non-compliance according to the judgment result, obviously provides a novel camera sensor installation compliance detection method, and can realize efficient and accurate detection judgment and early warning on the installation compliance of the camera sensor.
According to the installation data processing method based on the intelligent driving system, when whether the installation compliance condition is met or not is judged, according to three specific preset installation compliance conditions, the situation whether the installation compliance condition is met or not is judged, and the three situations are subdivided into three situations, namely pitch angle deviation judgment, deviation angle deviation judgment and roll angle deviation judgment, and respectively correspond to judgment on whether an image transverse horizontal line is coincided with a physical horizon or not, judgment on whether an image longitudinal central axis is coincided with a physical vehicle head perpendicular line or not and judgment on whether an image central point is coincided with an extended vanishing point or not. That is, if the image horizontal line does not coincide with the physical horizon, it will cause a pitch angle deviation; if the longitudinal central axis of the image is not coincident with the perpendicular bisector of the physical vehicle head, deviation angle deviation can be caused; if the image center point does not coincide with the extended vanishing point, it causes a roll angle deviation.
According to the installation data processing method based on the intelligent driving system provided by the invention, aiming at the pitch angle deviation judgment and/or the deviation angle deviation judgment, whether the camera sensor meets the preset installation compliance condition is judged according to the content of the driving image, fig. 2 is a second flow chart of the installation data processing method based on the intelligent driving system provided by the invention, and specifically as shown in fig. 2, the method is based on the method shown in fig. 1, step 200 is used for judging whether the camera sensor meets the preset installation compliance condition according to the content of the driving image, further, a target detection frame judgment method can be used, and the step 200 is understood as further comprising:
211. and judging whether the total number of the target detection frames in the driving image exceeds a first preset threshold value.
The target detection frame is determined according to a target detection model, and the target detection model is obtained by taking other small motor vehicles as detection targets and combining with some open source deep learning algorithms through pre-training. And the first preset threshold is a number threshold set for the total number of the target detection frames, for example, the threshold is set to be 4.
And judging whether the total number of the target detection frames in the driving image exceeds a first preset threshold value of 4, and when the total number of the target detection frames is more than 4, continuously monitoring all the target detection frames.
212. And when the total number of the target detection frames exceeds the first preset threshold value, detecting whether the upper edge line and the lower edge line of each target detection frame are in a first preset fluctuation range relative to the physical horizon.
When the total number of the target detection frames exceeds 4 of the first preset threshold, whether the upper and lower edge lines of each target detection frame are within a first preset fluctuation range relative to the physical horizon is also detected, the first preset fluctuation range refers to a range floating upward and downward relative to the physical horizon, in the driving image, in order to represent accuracy, no cm or mm is used as a unit, but an image pixel is used as a unit, and a specific first preset fluctuation range can be preset according to an actual scene, for example, the first preset fluctuation range is set as a range of 50 pixel units above and below the physical horizon. For each object detection frame, it is determined whether or not both of the upper edge line and the lower edge line thereof are within a range of 50 pixel units above and below the physical horizon.
213. And determining whether pitch angle deviation and/or offset angle deviation exists according to the proportion of the number of the target detection frames in the first preset fluctuation range relative to the physical horizon to the total number.
Counting the number of target detection frames with upper and lower edge lines in the range of 50 pixel units relative to the physical horizon, calculating the proportion of the number in the total number of the target detection frames, and if the proportion is greater than 1/2, judging that pitch angle deviation and offset angle deviation do not exist, namely that the installation compliance condition corresponding to the pitch angle deviation and the installation compliance condition corresponding to the offset angle deviation both accord with each other. If the ratio is 1/2 or less, it is determined that there is a pitch angle deviation and/or a yaw angle deviation, resulting in an improper mounting of the on-vehicle image sensor.
Of course, it may be further performed that the number of times of occurrence of the standard detection frames whose upper and lower edges are in the first preset fluctuation range within a period of time is counted, and the occurrence probability meeting the first preset fluctuation range is calculated according to all the target detection frames, and if the occurrence probability is greater than 1/2, it is determined that there is no pitch angle deviation and/or deviation angle deviation, that is, both the installation compliance condition corresponding to the pitch angle deviation and the installation compliance condition corresponding to the deviation angle deviation meet. If the probability is not more than 1/2, it is determined that there is a pitch angle deviation and/or a yaw angle deviation, resulting in an improper mounting of the on-vehicle image sensor.
Specifically, when it is determined that there is a pitch angle deviation and/or a deviation angle deviation, it is possible to further analyze, in the travel image content, the longitudinal deviation amount and/or the lateral deviation amount generated by the small-sized motor vehicle icon in the target detection frame to distinguish whether there is only a pitch angle deviation, only a deviation angle deviation, or both the pitch angle deviation and the deviation angle deviation, thereby determining the details of the installation irregularity. And according to the detailed situation of non-compliance, after the early warning information of non-compliance installation is sent, reference basis can be provided for relevant personnel, so that the relevant personnel can effectively adjust the installation angle and the position of the camera sensor according to the reference basis, the installation correction of the camera sensor is realized, and the camera sensor meets the installation compliance.
According to the installation data processing method based on the intelligent driving system, the target detection frame is determined according to a target detection model which takes other small motor vehicles in the driving image as detection targets and is trained by combining a YOLO open source algorithm and/or an SSD open source algorithm.
The target detection model is a model which is trained in advance by taking other small motor vehicles in the driving image as detection targets and combining a YOLO open source algorithm and/or an SSD open source algorithm. Small motor vehicles such as compact vehicles, SUVs, ordinary vans, and cars, which are suitably sized as detection targets, do not include pedestrians, and do not include large vehicles such as trucks and buses. When the target detection model is trained in advance, a large number of images of small motor vehicles are collected as a data set, the image content is subjected to real-time reasoning and practicing by combining a YOLO open source algorithm and/or an SSD open source algorithm and the like, and the target detection model is finally trained, wherein the specific content relates to the type of a vehicle to which a detection target belongs, the position and the size of the detection target in the images and the like. Therefore, the target detection frame required in the current detection process can be quickly and accurately determined according to the target detection model.
According to the installation data processing method based on the intelligent driving system provided by the present invention, for the pitch angle deviation judgment and/or the offset angle deviation judgment and/or the roll angle deviation judgment, whether the camera sensor meets the preset installation compliance condition is judged according to the content of the driving image, fig. 3 is a third flow chart of the installation data processing method based on the intelligent driving system provided by the present invention, specifically as shown in fig. 3, on the basis of the method shown in fig. 1, step 200 judges whether the camera sensor meets the preset installation compliance condition according to the content of the driving image, and further includes:
221. and judging whether the running speed of the current vehicle exceeds a second preset threshold value.
The second preset threshold is a speed threshold preset for the running speed of the current vehicle, and may be specifically set to 60Km/h, that is, it is determined whether the real-time running speed of the current vehicle exceeds the second preset threshold of 60 Km/h. When the vehicle speed is too slow, the vehicle runs very stably, the vehicle-mounted camera sensor does not swing very much, and the judgment meaning is not great at the moment; when the vehicle reaches a certain speed, the slight angle or position change of the camera sensor is more obvious, and the judgment significance is greater.
222. And when the running vehicle speed exceeds the second preset threshold value, determining a vanishing point position by a vanishing point position judging method, and judging whether the vanishing point position is in a second preset fluctuation range around the image central point.
And when the real-time running speed of the current vehicle exceeds the second preset threshold value of 60Km/h, determining a vanishing point position in the running image by a vanishing point position judging method, and judging whether the vanishing point position is in a second preset fluctuation range around the image center point of the running image. The second predetermined fluctuation range is a circular range with a certain radius value outward from the image center point, for example, a circular range with a radius of 50 pixel units outward from the image center point, and it is determined whether the determined specific vanishing point position falls within the predetermined circular range.
223. It is determined whether a pitch angle deviation and/or a yaw angle deviation and/or a roll angle deviation is present.
And judging whether the determined specific vanishing point position falls into a preset circular range or not, and further determining that pitch angle deviation, deviation angle deviation and roll angle deviation do not exist if the specific vanishing point position falls into the preset circular range, and installing and conforming the vehicle-mounted camera sensor. If the specific vanishing point position does not fall into the preset circular range, pitch angle deviation and/or offset angle deviation and/or roll angle deviation may exist, and the vehicle-mounted camera sensor is installed in an irregular manner.
Of course, it is also possible to further count the number of occurrences of the image center point falling within the preset circular range over a period of time, and calculate the occurrence probability meeting the second preset fluctuation range according to all the situations, and if the occurrence probability is greater than 1/2, it is also determined that the pitch angle deviation and/or the yaw angle deviation and/or the roll angle does not exist, that is, the installation compliance condition corresponding to the pitch angle deviation and the installation compliance condition corresponding to the yaw angle deviation and the installation compliance condition corresponding to the roll angle deviation both meet. If the probability is not more than 1/2, it is determined that there is a pitch angle deviation, a yaw angle deviation, and a roll angle deviation, and the vehicle-mounted imaging sensor is not installed properly.
When pitch angle deviation and/or deviation angle deviation and/or roll angle deviation exists, the specific vanishing point position can be analyzed by combining with other image contents in the driving image, in a coordinate system formed by taking the current image central point as a coordinate origin and presetting the circular range, the specific vanishing point position is in which quadrant and specific position relative to the coordinate origin, and according to the pixel coordinates of the quadrant and the specific position, whether only one deviation among the pitch angle deviation, the deviation angle deviation and the roll angle deviation exists or whether two deviations exist or whether three deviations exist in an unqualified detailed situation is determined. And according to the detailed situation of non-compliance, after the early warning information of non-compliance installation is sent, reference basis can be provided for relevant personnel, so that the relevant personnel can effectively adjust the installation angle and the position of the camera sensor according to the reference basis, the installation correction of the camera sensor is realized, and the camera sensor meets the installation compliance.
Further, the determination process for the non-compliant situation of the pitch angle deviation and the offset angle deviation may be a comprehensive determination method combining the target detection frame determination method and the vanishing point position determination method, that is, the step 200 specifically includes the step 211 of the target detection frame determination method and the step 221 of the vanishing point position determination method, and the comprehensive determination method after combining the two methods finally combines the determination results of the target detection frame determination method and the vanishing point position determination method, and if any one of the determination result of the target detection frame determination method and the determination result of the vanishing point position determination method is that there is no pitch angle deviation and/or offset angle deviation, it is determined that the condition that the image horizontal line and the physical horizon line are coincident and/or the condition that the image longitudinal central axis and the physical vehicle head perpendicular line are coincident are met. And for the judging process of the unconventional situation of the lateral rolling angle deviation, only a vanishing point position judging method is used for judging, namely, the method of the step 221-223 is adopted for judging, when the central point of the image is determined to fall into the circular preset range according to the judging result, the lateral rolling angle deviation does not exist, and the condition that the central point of the image and the prolonged vanishing point coincide is met.
In other words, the target detection frame determination method and the vanishing point position determination method may be parallel to each other for the compliance condition determination of the pitch angle deviation/the offset angle deviation, and may be satisfied. The combination of the target detection frame judgment method and the vanishing point position judgment method can ensure that the comprehensive judgment result is more accurate and can be more suitable for different environments, such as: when the lane line cannot be detected in the driving image, the vanishing point position judgment method may be inaccurate, and at this time, a target detection frame judgment method may be used; if there are very few front small motor vehicles, the target detection frame determination method may be inaccurate, and at this time, the vanishing point position determination method may be used. However, only the vanishing point position determination method can be applied to the determination of the compliance condition of the roll angle deviation. It should be noted that the vehicle-mounted camera sensor installation compliance can be generally determined only when the conditions that the horizontal line of the image coincides with the physical horizon, the longitudinal central axis of the image coincides with the perpendicular bisector of the physical vehicle head, and the central point of the image coincides with the extended vanishing point must be met at the same time.
According to the installation data processing method based on the intelligent driving system, provided by the invention, the vanishing point position is determined by a vanishing point position judging method, and the method further comprises the following steps:
2221. and setting an interested area for the driving image according to the target view range.
The method includes the steps of determining a target view field (a road section interval and a view field space which are expected to be observed) in advance according to observation requirements of an actual scene, and selecting an interested region for a driving image of the current vehicle on the current road section according to the target view field, wherein the driving image is acquired, for example, the view field of 3 lanes in front of the vehicle is selected as the interested region.
2222. At least two lane lines are detected from the region of interest of the driving image by Canny edge detection and/or hough line detection.
And detecting at least two lane lines (the lane lines are straight lines) from the region of interest of the driving image according to a lane line detection method (Canny edge detection method) based on deep learning and/or a Hough line detection method.
2223. And performing operation processing on the detected at least two lane lines through a Randac algorithm to determine a vanishing point position.
And then, at least two lane lines detected in 2222 are subjected to operation processing through a Randac algorithm, and the vanishing point position is estimated.
And then, whether the specific vanishing point position is in a second preset fluctuation range around the central point of the image can be continuously judged.
According to the installation data processing method based on the intelligent driving system, whether the camera sensor meets the preset installation compliance condition or not is judged regularly according to the content of the driving image according to the preset time interval.
It may be set that the judgment of the installation compliance is made periodically every half hour.
The invention also provides an installation data processing device based on the intelligent driving system, the device corresponds to the installation data processing method based on the intelligent driving system, the application principles can be mutually referred, and the description is omitted.
The invention provides an installation data processing device based on an intelligent driving system, fig. 4 is a schematic structural diagram of the installation data processing device based on the intelligent driving system, as shown in fig. 4, the device comprises an acquisition module 410, a judgment module 420 and an early warning module 430 which are connected in sequence, wherein:
an obtaining module 410, configured to obtain a driving image of a current vehicle, which is acquired by a vehicle-mounted camera sensor;
a determining module 420, configured to determine whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, where the preset installation compliance condition at least includes: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point;
and the early warning module 430 is configured to send out early warning information that the installation is not compliant if it is determined that the camera sensor does not comply with the preset installation compliance condition.
The installation data processing device based on the intelligent driving system comprises an acquisition module 410, a judgment module 420 and an early warning module 430 which are sequentially connected, and all the modules are mutually matched to work, so that the device can realize efficient and accurate detection judgment and early warning on the installation compliance of a camera sensor.
Fig. 5 is a schematic structural diagram of the electronic device provided in the present invention, and as shown in fig. 5, the electronic device may include: a processor (processor)610, a communication Interface 620, a memory (memory)630 and a communication bus 540, wherein the processor 510, the communication Interface 520 and the memory 530 are communicated with each other via the communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform all or a portion of the steps of the intelligent driving system-based installation data processing method, which includes:
acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor;
judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point;
and if the camera shooting sensor is judged not to be in accordance with the preset installation compliance condition, sending out early warning information of installation non-compliance. Furthermore, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part of the technical solution that substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the installation data processing method based on the intelligent driving system according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing all or part of the steps of the installation data processing method based on an intelligent driving system provided in the above embodiments, the method including:
acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor;
judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point;
and if the camera shooting sensor is judged not to be in accordance with the preset installation compliance condition, sending out early warning information of installation non-compliance.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements all or part of the steps of the installation data processing method based on an intelligent driving system according to the above embodiments, the method including:
acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor;
judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point;
and if the camera shooting sensor is judged not to be in accordance with the preset installation compliance condition, sending out early warning information of installation non-compliance.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions may be essentially or partially implemented in the form of software products, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the index monitoring method according to the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. An installation data processing method based on an intelligent driving system is characterized by comprising the following steps:
acquiring a current vehicle running image acquired by a vehicle-mounted camera sensor;
judging whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, wherein the preset installation compliance condition at least comprises the following steps: the horizontal line of the vehicle icon in the image is coincided with the physical horizon, the longitudinal central axis of the vehicle icon in the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the vehicle icon in the image is coincided with the extended vanishing point;
if the camera sensor is judged not to meet the preset installation compliance condition, sending out early warning information of installation non-compliance;
judging whether the camera sensor meets a preset installation compliance condition or not according to the content of the driving image aiming at the pitch angle deviation and/or the deviation angle deviation, and comprising the following steps:
judging whether the total number of the target detection frames in the driving image exceeds a first preset threshold value or not;
when the total number of the target detection frames exceeds the first preset threshold value, detecting whether the upper edge line and the lower edge line of each target detection frame are in a first preset fluctuation range relative to the physical horizon;
and determining whether pitch angle deviation and/or deviation angle deviation exists according to the proportion of the number of the target detection frames in the first preset fluctuation range relative to the physical horizon to the total number and by combining the longitudinal deviation and/or the transverse deviation generated by the vehicle icons in the target detection frames.
2. The intelligent driving system-based installation data processing method according to claim 1, wherein the target detection frame is determined according to a target detection model which takes other small motor vehicles in the driving image as detection targets and is trained by combining a YOLO open source algorithm and/or an SSD open source algorithm.
3. The intelligent driving system-based installation data processing method according to claim 1, wherein the determining whether the camera sensor meets a preset installation compliance condition according to the content of the driving image for a pitch angle deviation determination and/or a yaw angle deviation determination and/or a roll angle deviation determination, further comprises:
judging whether the running speed of the current vehicle exceeds a second preset threshold value or not;
when the running speed exceeds the second preset threshold value, determining a vanishing point position by a vanishing point position judging method, and judging whether the vanishing point position is in a second preset fluctuation range around the image central point to obtain a vanishing point position judging result;
and determining whether pitch angle deviation and/or deviation angle deviation and/or roll angle deviation exists according to the vanishing point position judgment result and by combining the quadrant and position pixel coordinates of the vanishing point position in a coordinate system taking the current image central point as the coordinate origin and in a preset circular range.
4. The intelligent driving system-based installation data processing method according to claim 3, wherein the determining of the vanishing point position through the vanishing point position determination method comprises:
setting an interested area for the driving image according to the target view range;
detecting at least two lane lines from the region of interest of the driving image by Canny edge detection and/or hough line detection;
and performing operation processing on the detected at least two lane lines through a Randac algorithm to determine a vanishing point position.
5. The intelligent driving system-based installation data processing method according to claim 1, wherein whether the camera sensor meets a preset installation compliance condition is periodically determined according to the content of the driving image at preset time intervals.
6. An installation data processing device based on an intelligent driving system is characterized by comprising:
the acquisition module is used for acquiring a running image of the current vehicle acquired by the vehicle-mounted camera sensor;
a judging module, configured to judge whether the camera sensor meets a preset installation compliance condition according to the content of the driving image, where the preset installation compliance condition at least includes: the horizontal image horizontal line is coincided with the physical horizon line, the longitudinal central axis of the image is coincided with the perpendicular bisector of the physical vehicle head, and the central point of the image is coincided with the extended vanishing point; the judging module is specifically used for judging whether the total number of the target detection frames in the driving image exceeds a first preset threshold value; when the total number of the target detection frames exceeds the first preset threshold value, detecting whether the upper edge line and the lower edge line of each target detection frame are in a first preset fluctuation range relative to the physical horizon; determining whether pitch angle deviation and/or deviation angle deviation exists according to the proportion of the number of the target detection frames in the first preset fluctuation range relative to the physical horizon to the total number and by combining the longitudinal deviation and/or the transverse deviation generated by the vehicle icons in the target detection frames;
and the early warning module is used for sending out early warning information of installation failure if the camera sensor is judged not to meet the preset installation failure condition.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the computer program, implements all or part of the steps of the intelligent driving system-based installation data processing method according to any one of claims 1 to 5.
8. A computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing all or part of the steps of the installation data processing method based on an intelligent driving system according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110429222.5A CN112990117B (en) | 2021-04-21 | 2021-04-21 | Installation data processing method and device based on intelligent driving system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110429222.5A CN112990117B (en) | 2021-04-21 | 2021-04-21 | Installation data processing method and device based on intelligent driving system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112990117A CN112990117A (en) | 2021-06-18 |
| CN112990117B true CN112990117B (en) | 2021-08-17 |
Family
ID=76341437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110429222.5A Active CN112990117B (en) | 2021-04-21 | 2021-04-21 | Installation data processing method and device based on intelligent driving system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990117B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113588664A (en) * | 2021-08-02 | 2021-11-02 | 安徽省通途信息技术有限公司 | Vehicle-mounted road defect rapid inspection and analysis system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013001155A (en) * | 2011-06-13 | 2013-01-07 | Alpine Electronics Inc | Apparatus and method for detecting posture of on-vehicle camera |
| CN107194969A (en) * | 2012-10-16 | 2017-09-22 | 高通股份有限公司 | The pick up calibration determined based on end point and location estimation |
| CN107466385A (en) * | 2016-08-03 | 2017-12-12 | 深圳市大疆灵眸科技有限公司 | A kind of cloud platform control method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6354425B2 (en) * | 2014-07-30 | 2018-07-11 | 株式会社デンソー | In-vehicle camera mounting attitude detection method and apparatus |
| JP6565769B2 (en) * | 2016-04-03 | 2019-08-28 | 株式会社デンソー | In-vehicle camera mounting angle detection device, mounting angle calibration device, mounting angle detection method, mounting angle calibration method, and computer program |
| CN112215214A (en) * | 2020-12-11 | 2021-01-12 | 智道网联科技(北京)有限公司 | Method and system for adjusting camera offset of intelligent vehicle-mounted terminal |
-
2021
- 2021-04-21 CN CN202110429222.5A patent/CN112990117B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013001155A (en) * | 2011-06-13 | 2013-01-07 | Alpine Electronics Inc | Apparatus and method for detecting posture of on-vehicle camera |
| CN107194969A (en) * | 2012-10-16 | 2017-09-22 | 高通股份有限公司 | The pick up calibration determined based on end point and location estimation |
| CN107466385A (en) * | 2016-08-03 | 2017-12-12 | 深圳市大疆灵眸科技有限公司 | A kind of cloud platform control method and system |
Non-Patent Citations (2)
| Title |
|---|
| Estimate the attitude of the camera from the railway-environment images;Xiaoyue Luo etal.;《2014 12th International Conference on Signal Processing (ICSP)》;20150122;全文 * |
| 铁路运行环境下的车载相机姿态估计;于欣妍等;《光学技术》;20150531;第41卷(第3期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112990117A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111932901B (en) | Road vehicle tracking detection apparatus, method and storage medium | |
| CN112349144B (en) | Monocular vision-based vehicle collision early warning method and system | |
| CN110400478A (en) | A kind of road condition notification method and device | |
| CN110298307B (en) | Abnormal parking real-time detection method based on deep learning | |
| CN107097794B (en) | System and method for detecting lane line of road | |
| CN110942038A (en) | Traffic scene recognition method, device, medium and electronic equipment based on vision | |
| CN108482382B (en) | Driving technology scoring method, device, storage medium and vehicle | |
| CN111915883A (en) | Road traffic condition detection method based on vehicle-mounted camera shooting | |
| CN109703456B (en) | Warning method and device for preventing automobile collision and automobile controller | |
| CN109753841B (en) | Lane line identification method and device | |
| CN110341621B (en) | Obstacle detection method and device | |
| CN112990117B (en) | Installation data processing method and device based on intelligent driving system | |
| CN105022987A (en) | Method of misalignment correction and diagnostic function for lane sensing sensor | |
| CN113657265B (en) | Vehicle distance detection method, system, equipment and medium | |
| CN111931745B (en) | Vehicle detection method and device, electronic equipment and storage medium | |
| CN111967451B (en) | Road congestion detection method and device | |
| CN113380038A (en) | Vehicle dangerous behavior detection method, device and system | |
| CN112991769A (en) | Traffic volume investigation method and device based on video | |
| CN112562406A (en) | Method and device for identifying off-line driving | |
| CN113569666B (en) | Method for detecting continuous illegal lane change of vehicle and computer equipment | |
| CN116152753A (en) | Vehicle information identification method and system, storage medium and electronic device | |
| TWI621073B (en) | Road lane detection system and method thereof | |
| KR102418344B1 (en) | Traffic information analysis apparatus and method | |
| CN111874003B (en) | Vehicle driving deviation early warning method and system | |
| CN114078212A (en) | Accurate vehicle type identification method and device based on ETC portal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |