CN112906462A - Vehicle parking violation identification method, system and device - Google Patents
Vehicle parking violation identification method, system and device Download PDFInfo
- Publication number
- CN112906462A CN112906462A CN202110051239.1A CN202110051239A CN112906462A CN 112906462 A CN112906462 A CN 112906462A CN 202110051239 A CN202110051239 A CN 202110051239A CN 112906462 A CN112906462 A CN 112906462A
- Authority
- CN
- China
- Prior art keywords
- motion track
- dimensional model
- dimensional
- vehicle
- road section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a vehicle parking violation identification method, system and device. The technical scheme provided by the embodiment of the application constructs the three-dimensional model of the inspection road section based on the multi-line laser radar and the RTK positioning technology, and combines the three-dimensional model and the panoramic image data to carry out accurate identification of vehicle parking violation, reduces the complex flow of manual evidence obtaining identification, and improves the identification and management efficiency of vehicle parking violation.
Description
Technical Field
The embodiment of the application relates to the technical field of visual recognition, in particular to a vehicle parking violation recognition method, system and device.
Background
In order to manage and restrict illegal behaviors such as parking on a road and the like, the illegal behaviors of vehicles need to be identified and confirmed so as to cooperate with law enforcement of the illegal behaviors. At present, the main management means for the vehicle parking violation behaviors are divided into an on-site law enforcement means and an off-site law enforcement means, wherein the on-site law enforcement means mainly depends on a full-manual mode to perform evidence obtaining treatment, the law enforcement efficiency is low, the police force resource consumption is relatively large, and the situation of law enforcement personnel in 24 hours is difficult to ensure. Often, law enforcement personnel are orderly at the scene, and the phenomenon that the law enforcement personnel are disordered and still after leaving is difficult to play an obvious deterrent effect. The off-site law enforcement is mainly that law enforcement personnel manually control the pan tilt camera to zoom and turn, shoot vehicles in a proper scene, manually identify vehicle numbers, and manually recover the preset position of the camera after completion. The whole process is complex, labor cost is high, and law enforcement efficiency is low.
Disclosure of Invention
The embodiment of the application provides a vehicle illegal parking identification method, system and device, which can improve the management efficiency of illegal parking behaviors and guarantee the identification precision of the illegal parking behaviors of vehicles.
In a first aspect, an embodiment of the present application provides a vehicle parking violation identification method, including:
acquiring point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture, and extracting a first motion track of a system positioned under the inspection road section through RTK;
generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data;
determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix;
determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture;
extracting a two-dimensional map of the inspection road section, and superposing the second three-dimensional model to the two-dimensional map, wherein the two-dimensional map defines an illegal parking area in advance;
identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image;
and judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
Further, generating a first three-dimensional model corresponding to the relative coordinate system and a second motion track of the system on the patrol section according to the point cloud data, and the method comprises the following steps:
and constructing a first three-dimensional model of the inspection road section under a relative coordinate system by using a SLAM algorithm based on the point cloud data, and generating a second motion track of the system under the relative coordinate system on the inspection road section.
Further, determining a rotation-translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track in a geodetic coordinate system based on the rotation-translation transformation matrix, including:
registering the second motion track with the first motion track based on corresponding system time, and solving a rotation and translation transformation matrix of the second motion track and the first motion track;
and converting the first three-dimensional model under the relative coordinate system into a second three-dimensional model under the geodetic coordinate system through the rotation and translation transformation matrix, and converting the second motion track under the relative coordinate system into a third motion track under the geodetic coordinate system through the rotation and translation transformation matrix.
Further, determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion trajectory and based on the corresponding system time and the position posture, including:
selecting track points corresponding to each frame of image in the panoramic image data in the third motion track based on corresponding system time;
and determining the three-dimensional geographic coordinates of each pixel point of each frame of image in the second three-dimensional model based on the corresponding position posture and according to the track points.
Further, extracting the two-dimensional map of the routing inspection road section, and overlaying the second three-dimensional model on the two-dimensional map includes:
and extracting a pre-stored two-dimensional map of the routing inspection road section, and superposing the second three-dimensional model to the two-dimensional map based on the geographic coordinate information of the two-dimensional map and the three-dimensional geographic coordinate of the second three-dimensional model.
Further, the vehicle parking violation judgment is performed based on the comparison result between the coordinate area and the parking violation area, and includes:
and comparing whether the coordinate area is overlapped with the illegal parking area, if so, outputting a corresponding vehicle illegal parking report, wherein the vehicle illegal parking report comprises the target image and the panoramic image data acquired by the associated system time.
Further, after outputting the corresponding vehicle violation report, the method further includes:
and inputting the target image of the corresponding vehicle into a preset driver detection and identification model, and outputting a corresponding detection and identification result.
In a second aspect, an embodiment of the present application provides a vehicle parking violation identification system, including: the system comprises a multi-line laser radar, a panoramic camera, an RTK positioning device and a calculation processor;
the multi-line laser radar is used for acquiring point cloud data of the inspection road section;
the RTK positioning device is used for recording a first motion trail of a system on the inspection road section;
the panoramic camera is used for acquiring panoramic image data of the inspection road section;
the calculation processor is used for receiving the point cloud data, the first motion track and the image data, and generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data; determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix; determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture; extracting a two-dimensional map of the inspection road section, and superposing the second three-dimensional model to the two-dimensional map, wherein the two-dimensional map defines an illegal parking area in advance; identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image; and judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
In a third aspect, an embodiment of the present application provides a vehicle parking violation identification device, including:
the system comprises an acquisition module, a data acquisition module and a data acquisition module, wherein the acquisition module is used for acquiring point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture and extracting a first motion track of the system positioned under the inspection road section through RTK;
the generating module is used for generating a first three-dimensional model corresponding to the relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data;
the conversion module is used for determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix;
the first positioning module is used for determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture;
the superposition module is used for extracting a two-dimensional map of the routing inspection road section and superposing the second three-dimensional model to the two-dimensional map, and the two-dimensional map defines a parking violation area in advance;
the second positioning module is used for identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image;
and the judging module is used for judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
In a fourth aspect, an embodiment of the present application provides an electronic device, including:
a memory and one or more processors;
the memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the vehicle violation identification method according to the first aspect.
In a fifth aspect, the present embodiments provide a storage medium containing computer-executable instructions for performing the vehicle parking identification method according to the first aspect when executed by a computer processor.
The embodiment of the application collects point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture, extracts a first motion track of the system positioned by RTK under the inspection road section, generates a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system under the inspection road section according to the point cloud data, determines a rotation and translation transformation matrix of the second motion track and the first motion track, converts the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix, determines a three-dimensional geographic coordinate of each pixel point of the panoramic image data on the second three-dimensional model based on the third motion track and the corresponding system time and position posture, extracts a two-dimensional map of the inspection road section, superposes the second three-dimensional model on the two-dimensional map, and pre-demarcates a break area by the two-dimensional map, and identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image, and judging whether the vehicle is parked based on a comparison result of the coordinate area and the parking violation area. By adopting the technical means, the vehicle parking violation is accurately identified by combining the three-dimensional model and the panoramic image data, the complex process of manual evidence obtaining identification is reduced, and the vehicle parking violation identification and management efficiency is improved.
Drawings
FIG. 1 is a flowchart illustrating a method for recognizing a vehicle parking violation according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of a vehicle parking violation identification system according to an embodiment of the present application;
FIG. 3 is a flowchart of a rotational-translational transformation in accordance with a first embodiment of the present application;
fig. 4 is a flowchart of a pixel location method according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a vehicle parking violation identification apparatus according to a second embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to a third embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, specific embodiments of the present application will be described in detail with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some but not all of the relevant portions of the present application are shown in the drawings. Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations (or steps) as a sequential process, many of the operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The application provides a vehicle illegal parking identification method and system, and aims to achieve vehicle illegal parking identification through constructing a three-dimensional model of an inspection road section, correspondingly obtaining panoramic image data of the inspection road section, identifying and intercepting a vehicle image based on the panoramic image data, judging whether a vehicle is located in an illegal parking area or not based on the mapping relation between each pixel point of the panoramic image data and the three-dimensional model, and accordingly completing vehicle illegal parking identification. For the traditional vehicle illegal parking management mode, when the vehicle illegal parking management is carried out, no matter on-site law enforcement or off-site law enforcement is carried out, a large amount of manpower needs to be invested, the whole illegal parking management process is relatively complicated, and the evidence obtaining identification and the management efficiency of the vehicle illegal parking are relatively low. Based on this, the vehicle illegal parking identification method and system provided by the embodiment of the application are provided to solve the technical problems of complicated process and low efficiency of the existing vehicle illegal parking evidence obtaining identification process.
The first embodiment is as follows:
fig. 1 is a flowchart of a vehicle illegal parking identification method according to an embodiment of the present application, where the vehicle illegal parking identification method provided in this embodiment may be executed by a vehicle illegal parking identification device, the vehicle illegal parking identification device may be implemented by software and/or hardware, and the vehicle illegal parking identification device may be formed by two or more physical entities or may be formed by one physical entity. Generally, the vehicle violation identification device can be a computing device, such as a computing processor of a vehicle violation identification system.
The following description will be given taking a calculation processor of the vehicle parking violation identification system as an example of a main body for executing the vehicle parking violation identification method. Referring to fig. 1, the vehicle parking violation identification method specifically includes:
s110, collecting point cloud data and panoramic image data of the inspection road section based on corresponding system time and position postures, and extracting a first motion track of the system which is positioned under the inspection road section through RTK.
Illustratively, in the process of evidence obtaining and identifying the illegal vehicle parking, the illegal vehicle parking identification system provided by the embodiment of the application is used for patrolling a patrolling road section. In the routing inspection process, the system inspects point cloud data and panoramic image data of a road section in real time, and determines a first motion track of the system based on self positioning.
Specifically, referring to fig. 2, a schematic structural diagram of a vehicle parking violation identification system according to an embodiment of the present application is provided. The vehicle parking violation identification system comprises a multi-line laser radar, a panoramic camera, an RTK (Real-time kinematic) positioning device and a calculation processor; the multi-line laser radar is used for acquiring point cloud data of the inspection road section; the RTK positioning device is used for recording a first motion trail of a system on the inspection road section; the panoramic camera is used for acquiring panoramic image data of the inspection road section; the computing processor is used for receiving the point cloud data, the first motion track and the image data, and executing the vehicle illegal parking identification method based on the received related data, so that illegal parking identification and judgment of the vehicle are realized.
Furthermore, in the vehicle parking violation identification system provided by the embodiment of the application, the multi-line laser radar carries out point cloud data acquisition based on a relative coordinate system. And the three-dimensional geographic coordinates corresponding to all pixel points of the panoramic image data acquired by the panoramic camera are the three-dimensional geographic coordinates in a geodetic coordinate system. Similarly, for the first motion trajectory determined by the RTK positioning device, the three-dimensional geographic coordinates of the respective trace points are also the three-dimensional geographic coordinates in the geodetic coordinate system. Therefore, the embodiment of the application is different from the method that whether the vehicle is illegal and stopped is directly determined according to the mapping relation between the panoramic image data and the collected point cloud data, the three-dimensional model of the patrol road section under the earth coordinate system is determined through the conversion of the model, and then illegal and stopped identification of the vehicle is carried out based on the mapping relation between the three-dimensional model and the panoramic image data. Before that, a first three-dimensional model and a second motion track corresponding to a relative coordinate system (namely, a local coordinate system) are constructed based on point cloud data, and then the first three-dimensional model and the second motion track are converted into a second three-dimensional model and a third motion track corresponding to a geodetic coordinate system. And determining the mapping relation between the second three-dimensional model and each pixel point of the panoramic image data through the third motion trail. Based on the mapping relation, when the vehicle is detected by the panoramic image data and the vehicle is determined to just fall into the defined illegal parking area according to the second three-dimensional model, the current vehicle is judged to be illegal, so that the illegal parking judgment of the vehicle is accurately carried out, and the illegal parking identification and management efficiency of the vehicle is improved.
It should be noted that, when the multi-line lidar, the panoramic camera and the RTK positioning device acquire related data, system time for acquiring data of each part needs to be synchronously recorded, so as to facilitate subsequent model conversion and determination of three-dimensional geographic coordinates based on the system time. Moreover, the multiline laser radar, the panoramic camera and the RTK positioning device need to acquire related data based on the same position and posture, that is, the multiline laser radar, the panoramic camera and the RTK positioning device share one position posture. Therefore, the data correspondence of each part can be guaranteed, and the subsequent three-dimensional model conversion and coordinate determination are facilitated. In one embodiment, the relative pose relationship among the multi-line laser radar, the panoramic camera and the RTK positioning device can be determined, pose conversion is performed based on a standard position pose, the position poses of the multi-line laser radar, the panoramic camera and the RTK positioning device are unified, and data correspondence of all parts is guaranteed.
And S120, generating a first three-dimensional model corresponding to the relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data.
Further, for the collected point cloud data, since the point cloud data is obtained in a relative coordinate system (i.e., a local coordinate system), three-dimensional geographic coordinates corresponding to each pixel point of the panoramic image data need to be determined subsequently. Since the three-dimensional geographic coordinates use a geodetic coordinate system, a three-dimensional model under the corresponding geodetic coordinates needs to be obtained based on the point cloud data. Before that, a first three-dimensional model and a second movement track of the system on the patrol road section are generated based on the point cloud data.
Specifically, a first three-dimensional model of the routing inspection road section under a relative coordinate system is constructed by using a SLAM algorithm based on the point cloud data, and a second motion track of the system under the relative coordinate system on the routing inspection road section is generated. SLAM (simultaneous localization and mapping, instantaneous localization and mapping or simultaneous mapping and localization) can perform self-localization according to position estimation and a map in the system moving process, and build an incremental map on the basis of self-localization at the same time, so as to complete the construction of the first three-dimensional model. And determines a second motion profile based on the self-localization. The second motion trail comprises a series of trail points, and the trail points record corresponding system time, attitude angle information and three-dimensional coordinates under a relative coordinate system.
S130, determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track in a geodetic coordinate system based on the rotation and translation transformation matrix.
Based on the first three-dimensional model and the second motion trajectory generated in step S120, it is necessary to convert the first three-dimensional model and the second motion trajectory into a second three-dimensional model and a third motion trajectory in a geodetic coordinate system, so as to determine a mapping relationship between each pixel point of the panoramic image data and the second three-dimensional model. Wherein the second three-dimensional model and the third motion trajectory are generated by a rotational-translational transformation matrix, and referring to fig. 3, the rotational-translational transformation process includes:
s1301, registering the second motion track with the first motion track based on corresponding system time, and solving a rotation and translation transformation matrix of the second motion track and the first motion track;
s1302, the first three-dimensional model under the relative coordinate system is converted into a second three-dimensional model under the geodetic coordinate system through the rotation and translation transformation matrix, and the second motion track under the relative coordinate system is converted into a third motion track under the geodetic coordinate system through the rotation and translation transformation matrix.
Specifically, the first motion trail is obtained through an RTK positioning device, the RTK positioning device generates a first motion trail of the system in the geodetic coordinate through a differential GNSS (global navigation satellite system) technology, the first motion trail includes a series of track points, and each track point includes system time and a three-dimensional coordinate in the geodetic coordinate. And further registering the second motion track with the first motion track to obtain a rotation and translation transformation matrix. During registration, two track points with the same or the closest system time are respectively selected from the two motion tracks to be registered homonymous points, so that the three-dimensional coordinates of the two motion tracks are registered. In some embodiments, since the GNSS signals in the urban area may be blocked by tall buildings and trees, and errors may occur in a partial area in the first motion trajectory, a RANSAC (Random Sample Consensus) algorithm may be combined to perform registration of the two motion trajectories, so as to avoid the influence of the erroneous part.
Further, the formula of the rotational-translational transformation matrix is as follows:
X′＝R3×3X+T3×1 (1)
wherein X and X' are three-dimensional coordinates of the first motion track and the second motion track which are registered with the same point respectively,
in order to be a matrix of rotations,
is a translation matrix. Then, the rotation-translation transformation matrix of the second motion track and the first motion track can be obtained by a simultaneous equation by substituting the values of the two three-dimensional coordinates of X and X' into the above equations (1), (2) and (3).
Further, based on the obtained rotation-translation transformation matrix, the first three-dimensional model and the second motion track in the relative coordinate system can be converted into the second three-dimensional model and the third motion track in the geodetic coordinate system. It should be noted that, although the third motion trajectory and the first motion trajectory are both motion trajectories of the system corresponding to the inspection road section in the geodetic coordinate system, the third motion trajectory is converted from the second motion trajectory, and the trajectory points of the second motion trajectory contain corresponding attitude angle information, so that the third motion trajectory contains attitude angle information of each trajectory point relative to the first motion trajectory. The attitude angle information is used for determining the mapping relation between each pixel point of the subsequent panoramic image data and the second three-dimensional model.
And S140, determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture.
After the third motion trail and the second three-dimensional model are determined, positioning of each pixel point on the panoramic image data is carried out, and the corresponding three-dimensional coordinate is determined. Referring to fig. 4, the pixel location process includes:
s1401, selecting track points corresponding to each frame of image in the panoramic image data from the third motion track based on corresponding system time;
and S1402, determining the three-dimensional geographic coordinates of each pixel point of each frame of image in the second three-dimensional model based on the corresponding position posture and according to the track points.
It can be understood that, because the multiline laser radar and the panoramic camera of the vehicle parking violation identification system share one position and one attitude, the panoramic image data acquired at a certain track point of the third motion track should be mapped with the point cloud data acquired at the track point at the same system time. Based on this characteristic, frame images in the panoramic image data that match the respective trajectory points are determined at the same system time. And determining point cloud data which is acquired under the same position posture with the frame of image, namely the three-dimensional geographic coordinates corresponding to the second three-dimensional model, based on the track points corresponding to the frame of image. Therefore, the three-dimensional geographic coordinates of each pixel point in each frame of image in the second three-dimensional model, namely the mapping relation between each pixel point of the panoramic image data and the second three-dimensional model, are determined.
S150, extracting a two-dimensional map of the routing inspection road section, and overlaying the second three-dimensional model to the two-dimensional map, wherein the two-dimensional map defines an illegal parking area in advance.
And S160, identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image.
Specifically, the second three-dimensional model is converted from the first three-dimensional model constructed in real time. It cannot do the marking of the illegal regions in advance. The embodiment of the application establishes and stores the two-dimensional map of the patrol road section in advance, and defines the illegal parking area of the vehicle on the two-dimensional map. The second three-dimensional model is further superimposed onto the two-dimensional map. It will be appreciated that the two-dimensional map and the second three-dimensional model are two-dimensional and three-dimensional map data of the patrol section in the geodetic coordinate system, and the coordinates of the two-dimensional map and the second three-dimensional model should correspond to each other, and the three-dimensional coordinates of the second three-dimensional model only have information of the height part in comparison with the two-dimensional map. Therefore, by extracting the pre-stored two-dimensional map of the patrol inspection road section, the second three-dimensional model can be superposed on the two-dimensional map based on the corresponding relation between the geographic coordinate information of the two-dimensional map and the three-dimensional geographic coordinate of the second three-dimensional model.
Further, target detection and identification are carried out on the panoramic image data, and a target image containing a vehicle or a license plate is determined. And detecting and identifying the vehicle or the license plate in the panoramic image data by constructing a target detection and identification network model based on a convolutional neural network in advance. In the prior art, there are many techniques for performing target detection based on a target detection recognition model, and the specific detection recognition mode is not fixedly limited in the embodiment of the present application. And correspondingly detecting and identifying and intercepting the target image, wherein the three-dimensional coordinates of each pixel point in the panoramic image data in the second three-dimensional model are determined before. Then, based on the three-dimensional coordinates of each pixel point of the target image in the second three-dimensional model, the coordinate area of the target image in the second three-dimensional model can be determined. It will be appreciated that, since the two-dimensional map overlaps the second three-dimensional model, the coordinate region may also be correspondingly marked on the two-dimensional map.
S170, vehicle illegal parking judgment is carried out based on the comparison result of the coordinate area and the illegal parking area.
Finally, the coordinate area on the two-dimensional map is compared with the illegal parking area, and whether the vehicle corresponding to the coordinate area has illegal parking conditions can be judged. And comparing whether the coordinate area is overlapped with the illegal parking area, if so, outputting a corresponding vehicle illegal parking report, wherein the vehicle illegal parking report comprises the target image and the panoramic image data acquired by the associated system time. It can be understood that if the coordinate area and the parking violation area have an overlapping portion, it indicates that the current vehicle has a parking violation condition. Otherwise, no violation condition exists.
It should be noted that, when the vehicle is judged to be illegal, the corresponding vehicle illegal parking report is output, so that the current vehicle illegal parking situation is conveniently informed to the management personnel or the traffic law enforcement personnel. The management personnel or the traffic law enforcement personnel can visually and accurately determine whether the current vehicle has the condition of illegal parking based on the vehicle illegal parking report, the whole process does not need manual intervention to carry out illegal parking identification and evidence collection, great convenience is brought to the management personnel or the traffic law enforcement personnel, and illegal parking identification and management efficiency are improved.
And, the violation report includes the target image and panoramic image data acquired at the associated system time with the target avatar. The associated system time is the designated time adjacent to the system time corresponding to the target image, and the adjacent time period corresponding to the associated system time can be set according to actual needs. By providing panoramic image data of adjacent time, management personnel or traffic law enforcement personnel can further confirm the vehicle parking violation condition, and judgment errors are avoided.
Further, in one embodiment, the target image of the corresponding vehicle is input into a preset driver detection and recognition model, and the driver detection and recognition model is a target detection and recognition network model based on a convolutional neural network, and outputs a corresponding detection and recognition result through target detection and recognition. Through the detection and identification of the vehicle driver, the parking violation confirmation and evidence obtaining can be further carried out, and the rigor and the accuracy of the parking violation identification are ensured.
The method comprises the steps of collecting point cloud data and panoramic image data of an inspection road section based on corresponding system time and position postures, extracting a first motion track of the system positioned by RTK under the inspection road section, generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data, determining a rotation and translation transformation matrix of the second motion track and the first motion track, converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix, determining a three-dimensional geographic coordinate of each pixel point of the panoramic image data on the second three-dimensional model based on the third motion track and the corresponding system time and position postures, extracting a two-dimensional map of the inspection road section, superposing the second three-dimensional model to the two-dimensional map, and delimiting a break-stop area in advance by the two-dimensional, and identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image, and judging whether the vehicle is parked based on a comparison result of the coordinate area and the parking violation area. By adopting the technical means, the vehicle parking violation is accurately identified by combining the three-dimensional model and the panoramic image data, the complex process of manual evidence obtaining identification is reduced, and the vehicle parking violation identification and management efficiency is improved.
Example two:
on the basis of the foregoing embodiment, fig. 5 is a schematic structural diagram of a vehicle parking violation identification device according to a second embodiment of the present application. Referring to fig. 5, the vehicle parking violation identification apparatus provided in this embodiment specifically includes: the device comprises an acquisition module 21, a generation module 22, a conversion module 23, a first positioning module 24, a superposition module 25, a second positioning module 26 and a judgment module 27.
The acquisition module 21 is used for acquiring point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture, and extracting a first motion track of the system positioned under the inspection road section through RTK;
the generating module 22 is used for generating a first three-dimensional model corresponding to the relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data;
the conversion module 23 is configured to determine a rotation-translation transformation matrix of the second motion trajectory and the first motion trajectory, and convert the first three-dimensional model and the second motion trajectory into a second three-dimensional model and a third motion trajectory in a geodetic coordinate system based on the rotation-translation transformation matrix;
the first positioning module 24 is configured to determine, through the third motion trajectory and based on the corresponding system time and the position pose, a three-dimensional geographic coordinate of each pixel point of the panoramic image data in the second three-dimensional model;
the superposition module 25 is used for extracting a two-dimensional map of the routing inspection road section, superposing the second three-dimensional model to the two-dimensional map, and delimiting a parking violation area in advance by the two-dimensional map;
the second positioning module 26 is configured to identify and intercept a target image of a corresponding vehicle or license plate in the panoramic image, and determine a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image;
the judgment module 27 is configured to perform vehicle parking violation judgment based on a comparison result between the coordinate area and the parking violation area.
The method comprises the steps of collecting point cloud data and panoramic image data of an inspection road section based on corresponding system time and position postures, extracting a first motion track of the system positioned by RTK under the inspection road section, generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data, determining a rotation and translation transformation matrix of the second motion track and the first motion track, converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix, determining a three-dimensional geographic coordinate of each pixel point of the panoramic image data on the second three-dimensional model based on the third motion track and the corresponding system time and position postures, extracting a two-dimensional map of the inspection road section, superposing the second three-dimensional model to the two-dimensional map, and delimiting a break-stop area in advance by the two-dimensional, and identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image, and judging whether the vehicle is parked based on a comparison result of the coordinate area and the parking violation area. By adopting the technical means, the vehicle parking violation is accurately identified by combining the three-dimensional model and the panoramic image data, the complex process of manual evidence obtaining identification is reduced, and the vehicle parking violation identification and management efficiency is improved.
The vehicle illegal parking identification device provided by the second embodiment of the application can be used for executing the vehicle illegal parking identification method provided by the first embodiment of the application, and has corresponding functions and beneficial effects.
Example three:
an embodiment of the present application provides an electronic device, and with reference to fig. 6, the electronic device includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The number of processors in the electronic device may be one or more, and the number of memories in the electronic device may be one or more. The processor, memory, communication module, input device, and output device of the electronic device may be connected by a bus or other means.
The memory 32 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the vehicle parking violation identification method according to any embodiment of the present application (for example, the acquisition module, the generation module, the conversion module, the first positioning module, the superposition module, the second positioning module, and the judgment module in the vehicle parking violation identification device). The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system and an application program required by at least one function; the storage data area may store data created according to use of the device, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory may further include memory located remotely from the processor, and these remote memories may be connected to the device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The communication module 33 is used for data transmission.
The processor 31 executes various functional applications and data processing of the device by executing software programs, instructions and modules stored in the memory, that is, implements the above-described vehicle violation identification method.
The input device 34 may be used to receive entered numeric or character information and to generate key signal inputs relating to user settings and function controls of the apparatus. The output device 35 may include a display device such as a display screen.
The electronic device provided above can be used to execute the vehicle parking violation identification method provided in the first embodiment above, and has corresponding functions and beneficial effects.
Example four:
embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a vehicle parking violation identification method, including: acquiring point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture, and extracting a first motion track of a system positioned under the inspection road section through RTK; generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data; determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix; determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture; extracting a two-dimensional map of the inspection road section, and superposing the second three-dimensional model to the two-dimensional map, wherein the two-dimensional map defines an illegal parking area in advance; identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image; and judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
Storage medium-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDORAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system connected to the first computer system through a network (such as the internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations, e.g., in different computer systems connected by a network. The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.
Of course, the storage medium provided in the embodiments of the present application contains computer-executable instructions, and the computer-executable instructions are not limited to the vehicle parking violation identification method described above, and may also perform related operations in the vehicle parking violation identification method provided in any embodiment of the present application.
The vehicle parking violation identification device, the storage medium, and the electronic device provided in the above embodiments may execute the vehicle parking violation identification method provided in any embodiment of the present application, and the technical details not described in detail in the above embodiments may be referred to the vehicle parking violation identification method provided in any embodiment of the present application.
The foregoing is considered as illustrative of the preferred embodiments of the invention and the technical principles employed. The present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the claims.
Claims (11)
1. A vehicle parking violation identification method is characterized by comprising the following steps:
acquiring point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture, and extracting a first motion track of a system positioned under the inspection road section through RTK;
generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data;
determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix;
determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture;
extracting a two-dimensional map of the inspection road section, and superposing the second three-dimensional model to the two-dimensional map, wherein the two-dimensional map defines an illegal parking area in advance;
identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image;
and judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
2. The vehicle parking violation identification method according to claim 1, wherein generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the patrol inspection section according to the point cloud data comprises:
and constructing a first three-dimensional model of the inspection road section under a relative coordinate system by using a SLAM algorithm based on the point cloud data, and generating a second motion track of the system under the relative coordinate system on the inspection road section.
3. The vehicle parking violation identification method according to claim 1, wherein determining a roto-translational transformation matrix of the second motion trajectory and the first motion trajectory, and converting the first three-dimensional model and the second motion trajectory into a second three-dimensional model and a third motion trajectory in a geodetic coordinate system based on the roto-translational transformation matrix comprises:
registering the second motion track with the first motion track based on corresponding system time, and solving a rotation and translation transformation matrix of the second motion track and the first motion track;
and converting the first three-dimensional model under the relative coordinate system into a second three-dimensional model under the geodetic coordinate system through the rotation and translation transformation matrix, and converting the second motion track under the relative coordinate system into a third motion track under the geodetic coordinate system through the rotation and translation transformation matrix.
4. The vehicle parking violation identification method of claim 1, wherein determining three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion trajectory and based on the corresponding system time and position pose comprises:
selecting track points corresponding to each frame of image in the panoramic image data in the third motion track based on corresponding system time;
and determining the three-dimensional geographic coordinates of each pixel point of each frame of image in the second three-dimensional model based on the corresponding position posture and according to the track points.
5. The vehicle parking violation identification method according to claim 1, wherein extracting a two-dimensional map of the patrol inspection section and superimposing the second three-dimensional model on the two-dimensional map comprises:
and extracting a pre-stored two-dimensional map of the routing inspection road section, and superposing the second three-dimensional model to the two-dimensional map based on the geographic coordinate information of the two-dimensional map and the three-dimensional geographic coordinate of the second three-dimensional model.
6. The vehicle parking violation identification method according to claim 1, wherein the vehicle parking violation determination based on the comparison result between the coordinate area and the parking violation area comprises:
and comparing whether the coordinate area is overlapped with the illegal parking area, if so, outputting a corresponding vehicle illegal parking report, wherein the vehicle illegal parking report comprises the target image and the panoramic image data acquired by the associated system time.
7. The vehicle parking violation identification method according to claim 6, further comprising, after outputting the corresponding vehicle parking violation report:
and inputting the target image of the corresponding vehicle into a preset driver detection and identification model, and outputting a corresponding detection and identification result.
8. A vehicle parking violation identification system, comprising: the system comprises a multi-line laser radar, a panoramic camera, an RTK positioning device and a calculation processor;
the multi-line laser radar is used for acquiring point cloud data of the inspection road section;
the RTK positioning device is used for recording a first motion trail of a system on the inspection road section;
the panoramic camera is used for acquiring panoramic image data of the inspection road section;
the calculation processor is used for receiving the point cloud data, the first motion track and the image data, and generating a first three-dimensional model corresponding to a relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data; determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix; determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture; extracting a two-dimensional map of the inspection road section, and superposing the second three-dimensional model to the two-dimensional map, wherein the two-dimensional map defines an illegal parking area in advance; identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image, and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image; and judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
9. A vehicle parking violation identification device, comprising:
the system comprises an acquisition module, a data acquisition module and a data acquisition module, wherein the acquisition module is used for acquiring point cloud data and panoramic image data of an inspection road section based on corresponding system time and position posture and extracting a first motion track of the system positioned under the inspection road section through RTK;
the generating module is used for generating a first three-dimensional model corresponding to the relative coordinate system and a second motion track of the system on the inspection road section according to the point cloud data;
the conversion module is used for determining a rotation and translation transformation matrix of the second motion track and the first motion track, and converting the first three-dimensional model and the second motion track into a second three-dimensional model and a third motion track under a geodetic coordinate system based on the rotation and translation transformation matrix;
the first positioning module is used for determining the three-dimensional geographic coordinates of each pixel point of the panoramic image data in the second three-dimensional model through the third motion track and based on the corresponding system time and the position posture;
the superposition module is used for extracting a two-dimensional map of the routing inspection road section and superposing the second three-dimensional model to the two-dimensional map, and the two-dimensional map defines a parking violation area in advance;
the second positioning module is used for identifying and intercepting a target image of a corresponding vehicle or license plate in the panoramic image and determining a coordinate area formed by three-dimensional geographic coordinates of each pixel point in the target image;
and the judging module is used for judging the vehicle parking violation based on the comparison result of the coordinate area and the parking violation area.
10. An electronic device, comprising:
a memory and one or more processors;
the memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the vehicle violation identification method of any of claims 1-7.
11. A storage medium containing computer-executable instructions for performing the vehicle violation identification method according to any of claims 1-7 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110051239.1A CN112906462B (en) | 2021-01-14 | 2021-01-14 | Vehicle illegal parking identification method, system and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110051239.1A CN112906462B (en) | 2021-01-14 | 2021-01-14 | Vehicle illegal parking identification method, system and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112906462A true CN112906462A (en) | 2021-06-04 |
| CN112906462B CN112906462B (en) | 2023-03-31 |
Family
ID=76113220
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110051239.1A Active CN112906462B (en) | 2021-01-14 | 2021-01-14 | Vehicle illegal parking identification method, system and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112906462B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110223511A (en) * | 2019-04-29 | 2019-09-10 | 合刃科技（武汉）有限公司 | A kind of automobile roadside is separated to stop intelligent monitoring method and system |
| CN111275978A (en) * | 2019-12-23 | 2020-06-12 | 江西山水光电科技股份有限公司 | Roadside parking vehicle real-time acquisition management system and management method |
| CN111444845A (en) * | 2020-03-26 | 2020-07-24 | 江苏集萃华科智能装备科技有限公司 | Non-motor vehicle illegal parking identification method, device and system |
| CN111578957A (en) * | 2020-05-07 | 2020-08-25 | 泉州装备制造研究所 | Intelligent pure vehicle tracking and tracking method based on three-dimensional point cloud map positioning |
-
2021
- 2021-01-14 CN CN202110051239.1A patent/CN112906462B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110223511A (en) * | 2019-04-29 | 2019-09-10 | 合刃科技（武汉）有限公司 | A kind of automobile roadside is separated to stop intelligent monitoring method and system |
| CN111275978A (en) * | 2019-12-23 | 2020-06-12 | 江西山水光电科技股份有限公司 | Roadside parking vehicle real-time acquisition management system and management method |
| CN111444845A (en) * | 2020-03-26 | 2020-07-24 | 江苏集萃华科智能装备科技有限公司 | Non-motor vehicle illegal parking identification method, device and system |
| CN111578957A (en) * | 2020-05-07 | 2020-08-25 | 泉州装备制造研究所 | Intelligent pure vehicle tracking and tracking method based on three-dimensional point cloud map positioning |
Non-Patent Citations (1)
| Title |
|---|
| 倪志康等: "基于三维激光雷达与RTK融合的SLAM研究", 《制造业自动化》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112906462B (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111830953B (en) | Vehicle self-positioning method, device and system | |
| CN105674993A (en) | Binocular camera-based high-precision visual sense positioning map generation system and method | |
| US10872246B2 (en) | Vehicle lane detection system | |
| CN112836737A (en) | Roadside combined sensing equipment online calibration method based on vehicle-road data fusion | |
| CN109596121B (en) | Automatic target detection and space positioning method for mobile station | |
| CN113379805A (en) | Multi-information resource fusion processing method for traffic nodes | |
| CN109300143B (en) | Method, device and equipment for determining motion vector field, storage medium and vehicle | |
| CN112308913B (en) | Vehicle positioning method and device based on vision and vehicle-mounted terminal | |
| WO2011043275A1 (en) | Three-dimensional data creating method and three-dimensional data creating device | |
| WO2023123837A1 (en) | Map generation method and apparatus, electronic device, and storage medium | |
| CN109871739B (en) | Automatic target detection and space positioning method for mobile station based on YOLO-SIOCTL | |
| Zhou et al. | Developing and testing robust autonomy: The university of sydney campus data set | |
| CN114037762B (en) | Real-time high-precision positioning method based on registration of image and high-precision map | |
| Maanpää et al. | Multimodal end-to-end learning for autonomous steering in adverse road and weather conditions | |
| CN115900712A (en) | Information source reliability evaluation combined positioning method | |
| CN116958935A (en) | Multi-view-based target positioning method, device, equipment and medium | |
| Chen et al. | High-precision positioning, perception and safe navigation for automated heavy-duty mining trucks | |
| Mounier et al. | High-precision positioning in GNSS-challenged environments: a LiDAR-based multi-sensor fusion approach with 3D digital maps registration | |
| CN117635683A (en) | Trolley indoor positioning method based on multiple cameras | |
| CN117173214A (en) | High-precision map real-time global positioning tracking method based on road side monocular camera | |
| CN112906462B (en) | Vehicle illegal parking identification method, system and device | |
| CN116892949A (en) | Ground object detection device, ground object detection method, and computer program for ground object detection | |
| Certad et al. | IAMCV Multi-Scenario Vehicle Interaction Dataset | |
| Velat et al. | Vision based vehicle localization for autonomous navigation | |
| Certad et al. | Interaction of Autonomous and Manually Controlled Vehicles Multiscenario Vehicle Interaction Dataset |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 306, zone 2, building 1, Fanshan entrepreneurship center, Panyu energy saving technology park, No. 832 Yingbin Road, Donghuan street, Panyu District, Guangzhou City, Guangdong Province Applicant after: Jiadu Technology Group Co.,Ltd. Address before: Room 306, zone 2, building 1, Fanshan entrepreneurship center, Panyu energy saving technology park, No. 832 Yingbin Road, Donghuan street, Panyu District, Guangzhou City, Guangdong Province Applicant before: PCI-SUNTEKTECH Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |