CN111666853A - Real-time vehicle violation detection method, device, equipment and storage medium - Google Patents

Real-time vehicle violation detection method, device, equipment and storage medium Download PDF

Info

Publication number
CN111666853A
CN111666853A CN202010470098.2A CN202010470098A CN111666853A CN 111666853 A CN111666853 A CN 111666853A CN 202010470098 A CN202010470098 A CN 202010470098A CN 111666853 A CN111666853 A CN 111666853A
Authority
CN
China
Prior art keywords
vehicle
image
frame image
lane
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010470098.2A
Other languages
Chinese (zh)
Other versions
CN111666853B (en
Inventor
芦文峰
刘伟超
郭倜颖
曾凡涛
陈远旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202010470098.2A priority Critical patent/CN111666853B/en
Publication of CN111666853A publication Critical patent/CN111666853A/en
Priority to PCT/CN2020/118438 priority patent/WO2021120776A1/en
Application granted granted Critical
Publication of CN111666853B publication Critical patent/CN111666853B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention provides a real-time vehicle violation detection method, a device, equipment and a storage medium. The method comprises the following steps: acquiring and caching a real-time video image; recognizing vehicles and lane lines of an ith frame image and an i-1 th frame image in the video image, generating an identity for recognizing the vehicles, and segmenting and matching the vehicles and the lanes of the ith frame image and the i-1 th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image; judging whether the violation exists according to the vehicle segmentation matching image and the lane segmentation matching image; if the violation is judged, extracting vehicle information and reporting the violation according to the ith frame image; and if the extraction of the vehicle information fails, acquiring the identity of the violation vehicle, caching the identity in the video image to acquire the vehicle information, and reporting the violation. The invention can achieve real-time automatic violation detection, and can be applied to smart traffic scenes, thereby promoting the construction of smart cities.

Description

Real-time vehicle violation detection method, device, equipment and storage medium
Technical Field
The invention relates to the technical field of video processing, in particular to a real-time vehicle violation detection method, a real-time vehicle violation detection device, real-time vehicle violation detection equipment and a storage medium.
Background
The intelligent traffic violation detection is to automatically judge the violation of the traffic regulations of the vehicle by using various sensors and image acquisition equipment and combining a violation judgment algorithm and an information extraction algorithm at the rear end. The technology can help traffic management departments to improve the speed and accuracy of violation judgment, reduce labor cost and avoid the deviation of false detection, missed detection and the like caused by human reasons.
The intelligent violation detection technology on the market at present is divided into two types: one is to install a hardware device for detecting the violation in a fixed position, use sensors such as light or pressure to sense information such as the position of a vehicle, and then combine a single or a plurality of pictures obtained by photographing to judge the violation and extract the information. The hardware device for detecting the violation can only be installed at a fixed position to detect the specific violation, is not flexible enough, and can only supervise the vehicle at the fixed position.
The other method is that a vehicle-mounted camera is used for acquiring a driving video, and then one frame or a plurality of frames in the video are processed locally or sent to a remote server for processing so as to judge the violation. Processing one or more frames in the video locally may cause difficulty in extracting vehicle information, such as the extracted image with the license plate blocked; the transmission to a remote server for processing increases the download burden of the server and cannot be used on a large scale.
Disclosure of Invention
The invention provides a real-time vehicle violation detection method, a real-time vehicle violation detection device, equipment and a storage device, which can achieve the purposes of real-time automatic violation detection and accurate extraction of violation vehicle information.
In order to solve the technical problems, the invention adopts a technical scheme that: the real-time vehicle violation detection method comprises the following steps:
acquiring a real-time video image;
caching the video image;
recognizing vehicles and lane lines of an ith frame image and an i-1 th frame image in the video image, generating identification marks for each recognized vehicle, and segmenting and matching the vehicles and lanes of the ith frame image and the i-1 th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2;
judging whether a vehicle breaks rules or not in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image;
if the vehicle violation is judged, extracting the vehicle information of the vehicle violating the regulations according to the ith frame image;
if the vehicle information is successfully extracted, the extracted vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image are sent for violation reporting;
if the vehicle information extraction fails, acquiring the identity of the violation vehicle; and
and acquiring corresponding vehicle information in the cached video image according to the identity mark, and sending the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image for violation reporting.
According to an embodiment of the present invention, the identifying the vehicle and lane line of the i-th frame image and the i-1 th frame image in the video image, generating the identification mark for each identified vehicle, and segmenting and matching the vehicle and lane of the i-th frame image and the i-1 th frame image according to the identification result to obtain the vehicle segmentation matching image and the lane segmentation matching image includes:
recognizing each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of solid line lane, guide line lane and bus lane;
generating an identity of each vehicle according to the vehicle attribute;
dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image;
carrying out vehicle matching on the ith frame image and the vehicle segmentation image of the (i-1) th frame image to obtain the vehicle segmentation matching image; and
and carrying out lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image.
According to an embodiment of the present invention, the vehicle segmentation matching image is obtained by performing vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1 th frame image, and the method includes:
comparing the similarity of each vehicle in the vehicle segmentation image of the i-1 frame image with each vehicle in the vehicle segmentation image of the i frame image to obtain a first similarity comparison result, wherein the first similarity comparison result indicates that each vehicle comprising the i-1 frame image and at least one vehicle of the i frame image are similar vehicles;
randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the vehicle in the i-th frame image or performing residual matching with the unmatched vehicle in the i-th frame image,
judging whether the residual matching result is reasonable or not according to the first similarity ratio pair result;
if the image is reasonable, splicing the vehicle segmentation images of the ith frame image and the (i-1) th frame image together and loading a matched vehicle on a matched line so as to obtain the vehicle segmentation matched image,
and if the vehicle is not reasonable, matching the vehicle of the i-1 th frame image with the other corresponding similar vehicle in the i-1 th frame image until the residual matching result is judged to be reasonable, splicing the vehicle segmentation images of the i-1 th frame image and the i-1 th frame image together, and loading a matched line on the matched vehicle so as to obtain the vehicle segmentation matching image.
According to an embodiment of the present invention, performing lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image includes:
carrying out similarity comparison on each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that each lane of the i-1 th frame image and at least one lane of the i-1 th frame image are similar lanes;
randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the lane in the i-th frame image or performing residual matching with the unmatched lane in the i-th frame image,
judging whether the residual matching result is reasonable or not according to the second similarity comparison result
If the lane segmentation images are reasonable, splicing the i-th frame image and the lane segmentation images of the i-1 th frame image together and loading matched lines on matched lanes to obtain the lane segmentation matched images,
and if the lane is not reasonable, matching the lane of the i-1 th frame image with the other corresponding similar lane in the i-1 th frame image until the residual matching result is judged to be reasonable, splicing the i-th frame image and the lane segmentation image of the i-1 th frame image together, and loading a matched line on the matched lane so as to obtain the lane segmentation matching image.
According to an embodiment of the present invention, the determining whether there is a vehicle violation in the i-th frame image according to the vehicle segmentation matching image and the lane segmentation matching image includes:
and judging whether a vehicle violates a regulation or not according to the lane attributes of each lane in the lane segmentation matching image and by combining a vehicle matching line in the vehicle segmentation matching image, and if the lane with the preset lane attribute in the lane segmentation matching image and the vehicle matching line extend to the lane with the preset lane attribute and other lanes, judging that the vehicle corresponding to the vehicle matching line is a violation vehicle.
According to an embodiment of the present invention, the caching the video image further includes:
deleting at least a portion of the video image that was first stored when the video image exceeds storage capacity; the at least part of the video image is a frame of video image.
According to an embodiment of the invention, the method further comprises: storing the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report; and uploading the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report to a remote server at intervals of predetermined time or when a preset condition is met, and deleting the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report after the uploading is finished.
In addition, in order to solve the above technical problems, the present invention further adopts a technical solution in which: there is provided a real-time vehicle violation detection apparatus, the apparatus comprising:
the video receiving module is used for acquiring a real-time video image;
the video caching module is used for caching the video image;
the information extraction module is used for recognizing vehicles and lane lines of the ith frame image and the (i-1) th frame image in the video image, generating an identity for each recognized vehicle, and segmenting and matching the vehicles and lanes of the ith frame image and the (i-1) th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2;
the violation judging module is used for judging whether a vehicle violates the regulations in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; when the fact that the vehicle breaks rules is judged, vehicle information of the vehicle breaking rules is extracted according to the ith frame image;
the violation information extraction module is used for acquiring the identity of the violation vehicle when the violation judgment module fails to extract the vehicle information of the violation vehicle; acquiring corresponding vehicle information in the cached video image according to the identity;
the violation information storage module is used for storing the vehicle information and the ith frame image or the vehicle information, the ith frame image and the (i-1) th frame image which are related to violation report;
and the violation information sending module is used for sending the vehicle information related to violation report and the ith frame image or the vehicle information, the ith frame image and the (i-1) th frame image to carry out violation report.
In addition, in order to solve the above technical problems, the present invention further adopts a technical solution in which: there is provided a real-time vehicle violation detection device comprising a processor, a memory coupled to the processor, wherein the memory has stored therein computer readable instructions that, when executed by the processor, cause the processor to perform the steps of the real-time vehicle violation detection method described above.
In addition, in order to solve the above technical problems, the present invention further adopts a technical solution in which: a storage medium is provided having computer readable instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform the steps of the real-time vehicle violation detection method described above.
The invention provides a real-time vehicle violation detection method, a device, equipment and a storage device, which can realize real-time automatic detection of vehicle violation and extraction of violation vehicle information by acquiring a video image in real time, sequentially carrying out vehicle and lane line identification on an ith frame image and an i-1 th frame image, generating an identity mark for each identified vehicle, carrying out vehicle and lane segmentation and matching on the ith frame image and the i-1 th frame image according to an identification result, obtaining a vehicle segmentation matching image and a lane segmentation matching image, judging whether the vehicle violation exists in the ith frame image according to the vehicle segmentation matching image and the lane segmentation matching image, extracting violation vehicle information from violation vehicles, realizing the real-time automatic detection of vehicle violation and the extraction of violation vehicle information, having high detection efficiency and identifying each frame without frame leakage, missing detection in the snapshot gap is avoided.
Furthermore, when the illegal vehicle information extraction fails, the corresponding vehicle information is obtained in the cached video image through the identity of the illegal vehicle, so that the accuracy and the effectiveness of the illegal vehicle information extraction are guaranteed.
And further, sequentially carrying out vehicle and lane line identification on the ith frame image and the (i-1) th frame image, and generating an identity of each vehicle for each identified vehicle according to the vehicle attribute, so that the identity of each vehicle in the video image is unique.
Furthermore, the similarity between each vehicle and each lane line in the vehicle segmentation image of the i-1 frame image and each vehicle and each lane line in the lane line segmentation image of the i-1 frame image and each lane line in the vehicle segmentation image of the i-1 frame image is compared with the similarity between the obtained vehicle segmentation matching image and the obtained lane line segmentation matching image, so that the algorithm is simple, the operation efficiency is high, and the vehicle and the lane line can be matched quickly and accurately.
Furthermore, whether the vehicle violates the regulations is judged according to the lane attributes of each lane in the lane segmentation matching image and by combining the vehicle matching line in the vehicle segmentation matching image, the automatic detection of the vehicle violations can be completed through one frame of image or two continuous frames of images, the detection efficiency is high, and the detection result is accurate.
Further, when the video image exceeds the storage capacity, at least part of the video image which is stored firstly is deleted, the real-time violation detection result is not influenced, and the storage space can be released.
Further, vehicle information related to violation reporting and the ith frame image or the ith frame image and the (i-1) th frame image are stored, and violation evidence can be effectively stored.
Further, the stored vehicle information related to violation reporting and the ith frame image or the ith frame image and the (i-1) th frame image are stored and uploaded to a remote server every predetermined time or when preset conditions are met, local storage is deleted after uploading is completed, violation evidence can be uploaded when the system is idle, uploading success rate is guaranteed, the local storage is deleted after uploading is successful, and local storage space is further released.
Drawings
FIG. 1 is a schematic flow diagram of a real-time vehicle violation detection method of one embodiment of the present invention;
FIG. 2 is a schematic flow chart of a vehicle matching process in one embodiment of the present invention;
FIG. 3 is a schematic diagram of the configuration of a real-time vehicle violation detection apparatus in accordance with one embodiment of the present invention;
FIG. 4 is a schematic diagram of the configuration of a real-time vehicle violation detection device in accordance with one embodiment of the present invention;
fig. 5 is a schematic structural diagram of a storage medium according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first", "second" and "third" in this application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any indication of the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of the feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless explicitly specifically limited otherwise. All directional indications (such as up, down, left, right, front, and rear … …) in the embodiments of the present application are only used to explain the relative positional relationship between the components, the movement, and the like in a specific posture (as shown in the drawings), and if the specific posture is changed, the directional indication is changed accordingly. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
FIG. 1 is a flow diagram of a real-time vehicle violation detection method of one embodiment of the present invention. The real-time vehicle violation detection method can be operated on a vehicle-mounted intelligent device, the vehicle-mounted intelligent device can be installed on a vehicle (such as a bus, a traffic assistant vehicle, a private car and the like), can acquire video images of lanes and vehicles at the front and the back, and can detect and detect vehicle violation in real time according to the video images. The application can also be applied to smart traffic scenes, so that the construction of smart cities is promoted. It should be noted that the method of the present invention is not limited to the flow sequence shown in fig. 1 if the results are substantially the same. As shown in fig. 1, the method comprises the steps of:
step S11: and acquiring a real-time video image.
It should be noted that, in this embodiment, the video image is obtained by shooting through the vehicle-mounted intelligent device in the driving process, and the vehicle-mounted intelligent device may be a shooting device with a driving shooting recording function, such as a vehicle-mounted driving recorder.
Step S12: and caching the video image.
In step S12, storing the video image in real time, and when the video image exceeds the storage capacity, deleting at least a part of the video image stored first; in this embodiment, at least a part of the video images may be one frame of video images, that is, when the storage capacity is full, every time a frame of the video images at the current time is newly stored, the oldest frame of the video images stored in the storage capacity is deleted.
Step S13: recognizing vehicles and lane lines of the ith frame image and the (i-1) th frame image in the video image, generating identification marks for each recognized vehicle, and segmenting and matching the vehicles and the lanes of the ith frame image and the (i-1) th frame image according to the recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2.
Specifically, step S13 includes the steps of:
step S131: and identifying each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of solid line, guide line and bus lane.
It can be understood that, in step S131, an image recognition technology may be adopted to respectively recognize a vehicle and a lane line as a target object for each frame of image, and each lane and its lane type, each vehicle and its vehicle attribute in each frame of image are obtained through recognition, in this embodiment, the vehicle attribute includes one, two or more of a color, a vehicle type, a brand, and a license plate, and the lane type includes one, two or more of a real-line lane, a diversion lane, and a bus lane.
Step S132: and generating the identity of each vehicle according to the vehicle attributes.
According to an embodiment of the present invention, the identity of each vehicle may be generated according to the vehicle attributes, for example, an attribute value and a weight are set for each attribute in the vehicle attributes, and the identity is calculated by the attribute value and the weight, for example: the vehicle attribute of one vehicle is red and car, the attribute value of "red" is set to 10, the weight is 0.4, the attribute value of "car" is set to 20, and the weight is 0.6, then the weight may be based on the attribute value, and the weight generates the id 10 × 0.4+20 × 0.6 ═ 16 for the vehicle.
Step S133: and dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image.
Through an image recognition technology, the vehicle and the lane line can be recognized in each frame of image respectively, and the vehicle and the lane line are segmented from the image to obtain the vehicle segmentation image and the lane segmentation image respectively.
Step S134: and carrying out vehicle matching on the ith frame image and the vehicle segmentation image of the (i-1) th frame image to obtain the vehicle segmentation matching image.
Specifically, similarity comparison is performed between each vehicle in the vehicle segmentation image of the i-1 th frame image and each vehicle in the vehicle segmentation image of the i-th frame image to obtain a first similarity comparison result, and the first similarity comparison result indicates that each vehicle including the i-1 th frame image and at least one vehicle of the i-th frame image are similar vehicles; in this embodiment, performing similarity comparison between each vehicle in the vehicle segmentation image of the i-1 th frame image and each vehicle in the vehicle segmentation image of the i-th frame image may obtain a first similarity comparison result according to the similarity of the vehicle attributes and each vehicle in each frame image identified in step S131 and the vehicle attributes thereof.
Randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the vehicle in the i-th frame image or performing residual matching with the unmatched vehicle in the i-th frame image; in this embodiment, each vehicle in the i-1 th frame image is randomly matched with one of the similar vehicles corresponding to the vehicle in the i-1 th frame image, and the result of the random matching needs to satisfy that the vehicle in the i-1 th frame image and the similar vehicle corresponding to the vehicle in the i-1 th frame image correspond to each other one by one, that is, the multiple vehicles in the i-1 th frame image cannot correspond to the same similar vehicle in the i-1 th frame image at the same time, the same vehicle in the i-1 th frame image cannot correspond to the multiple similar vehicles in the i-1 th frame image at the same time, or the remaining vehicles in the i-1 th frame image and the unmatched vehicle in the i-1 th frame image are subjected to remaining matching.
Further, whether the remaining matching result is reasonable is determined according to the first similarity comparison result, in this embodiment, whether the remaining matching result is reasonable is determined, that is, whether the remaining matching result between the remaining vehicles in the i-1 th frame image and the remaining matching result between the remaining vehicles in the i-th frame image and the remaining vehicles in the non-matching vehicle in the i-th frame image satisfies the first similarity comparison result, that is, the vehicle attribute of the remaining matching vehicle also satisfies the similarity relationship in the first similarity comparison result. If the image is reasonable, namely the first similarity comparison result is met, the i-th frame image and the vehicle segmentation images of the i-1 th frame image are spliced together, matched vehicles are loaded on matched lines, so that the vehicle segmentation matched image is obtained, if the image is not reasonable, the vehicle of the i-1 th frame image is matched with the other similar vehicle corresponding to the vehicle in the i-th frame image, and the vehicle segmentation images of the i-1 th frame image are spliced together and the matched vehicles are loaded on the matched lines until the residual matching result is judged to be reasonable, so that the vehicle segmentation matched image is obtained.
The vehicle matching process described above is described below as an example. Referring to fig. 2, fig. 2 is a schematic flowchart of a vehicle matching process in an embodiment of the present invention, in which each vehicle in the vehicle segmentation image of the i-1 th frame image is compared with each vehicle in the vehicle segmentation image of the i-1 th frame image in a similarity comparison manner, in this embodiment, a first similarity comparison result may be obtained by performing a similarity comparison manner on the vehicle attributes, where the first similarity comparison result includes that an x1 vehicle of the i-1 th frame image is similar to a y1 vehicle and a y2 vehicle of the i-1 th frame image, an x2 vehicle of the i-1 th frame image is similar to a y2 vehicle and a y3 vehicle of the i-1 th frame image, and an x3 vehicle of the i-1 th frame image is similar to a y1 vehicle of the current image, as shown in fig. 2 (a).
Further, randomly matching the vehicles x1, x2 and x3 of the i-1 frame image with the corresponding similar vehicles of the i-1 frame image or performing residual matching on the remaining vehicles of the i-1 frame image with the unmatched vehicles in the i-1 frame image, wherein the result of the random matching needs to satisfy the one-to-one correspondence between the vehicles of the i-1 frame image and the corresponding similar vehicles in the i-1 frame image, if the x1 vehicle randomly matches the unmatched y1 vehicle of the corresponding similar vehicles and the x2 vehicle matches the unmatched y2 vehicle of the corresponding similar vehicles, the x3 vehicle can only match the unmatched remaining vehicles y3, and the y3 vehicle matched with the unmatched remaining vehicle according to the first similarity comparison result x3 is not in the first similarity comparison result, the vehicle of the i-1 th frame image is matched with the other similar vehicle corresponding to the vehicle of the i-1 th frame image in the example when the vehicle is matched again until the remaining matching result is judged to be reasonable, as shown in fig. 2(b), matching the x1 vehicle with another vehicle of y1 and y2, e.g., x1 vehicle matched y2 vehicle, x2 matched another vehicle remaining unmatched and similar to x2, e.g., x2 vehicle matched y3 vehicle, x3 matched the remaining unmatched y1 vehicles that were similar to x3, namely, the similar matching of each vehicle in the ith frame image and the vehicle in the (i-1) th frame image is completed, and splicing the ith frame of image and the vehicle segmentation image of the (i-1) th frame of image together, and loading a matched vehicle on a matched line so as to obtain the vehicle segmentation matched image.
In the matching process, if the number of each vehicle in the vehicle segmented image of the i-1 th frame image is different from the number of each vehicle in the vehicle segmented image of the i-1 th frame image, if the number of the vehicles in the vehicle segmented image of the i-1 th frame image is x1, x2 and x3, and the number of the vehicles in the vehicle segmented image of the i-1 th frame image is y1 and y2, it can be considered that there is one vehicle leaving the shooting picture in the vehicle segmented image of the i-1 th frame image, and it is only necessary to ensure the similarity comparison and matching of the vehicles y1 and y2 in the vehicle segmented image of the i-1 th frame image in the matching process; if the vehicles in the vehicle segmentation image of the i-1 frame image are x1 and x2, and the vehicles in the vehicle segmentation image of the i-1 frame image are y1, y2 and y3, it can be considered that there is one vehicle in the i-1 frame image entering the shooting picture, and only the similarity comparison and matching of the vehicles x1 and x2 in the vehicle segmentation image of the i-1 frame image need to be ensured in the matching process, and the unmatched vehicles in the vehicle segmentation image of the i-1 frame image can be compared and matched in the vehicle segmentation image of the next frame, i.e. the i +1 frame image.
Step S135: and carrying out lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image.
The lane segmentation matching image obtaining process is similar to the vehicle segmentation matching image obtaining process in the step S134, that is, similarity comparison is performed between each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-th frame image, so as to obtain a second similarity comparison result, where the second similarity comparison result includes that each lane of the i-1 th frame image and at least one lane of the i-th frame image are similar lanes; in this embodiment, the similarity comparison between each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-th frame image may be performed according to each lane in each frame image identified in step S131 and the lane type thereof, and according to the lane type, a second similarity comparison result is obtained.
Randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the lane in the i-th frame image or performing residual matching with the unmatched lane in the i-th frame image; in this embodiment, each lane of the i-1 th frame image is randomly matched with one of the similar lanes corresponding to the i-th frame image, and the result of the random matching needs to satisfy that the lanes of the i-1 th frame image correspond to the similar lanes corresponding to the i-th frame image one by one, that is, multiple lanes of the i-1 th frame image cannot correspond to the same similar lane of the i-th frame image at the same time, the same lane of the i-1 th frame image cannot correspond to multiple similar lanes of the i-th frame image at the same time, or the remaining lanes of the i-1 th frame image and the unmatched lanes of the i-th frame image are subjected to remaining matching.
Further, whether the remaining matching result is reasonable is judged according to the second similarity comparison result, in this embodiment, whether the remaining matching result is reasonable is judged, that is, whether the remaining matching result of the remaining lane of the i-1 th frame image and the remaining matching result of the unmatched lane in the i-th frame image satisfy the second similarity comparison result, that is, the lane type of the remaining matched lane also satisfies the similarity relationship in the second similarity comparison result. If the lane segmentation matching image is reasonable, namely the second similarity comparison result is met, splicing the i-th frame image and the lane segmentation image of the i-1 th frame image together, loading a matched line on the matched lane to obtain the lane segmentation matching image, if the lane segmentation matching image is not reasonable, matching the lane of the i-1 th frame image with the other similar lane corresponding to the lane in the i-th frame image, and if the remaining matching result is judged to be reasonable, splicing the lane segmentation images of the i-th frame image and the i-1 th frame image together, loading the matched line on the matched lane to obtain the lane segmentation matching image.
The lane matching process is also described below as an example. And carrying out similarity comparison on each lane in the lane segmentation image of the i-1 frame image and each lane in the lane segmentation image of the i-1 frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that the m1 lane of the i-1 frame image is similar to the n1 lane and the n2 lane of the i-1 frame image, the m2 lane of the i-1 frame image is similar to the n2 lane and the n3 lane of the i-1 frame image, and the m3 lane of the i-1 frame image is similar to the n1 lane of the current image.
Randomly matching m1, m2 and m3 lanes of the i-1 th frame image with the corresponding similar lanes of the i-1 th frame image or performing residual matching on the remaining lanes of the i-1 th frame image with the unmatched lanes of the i-1 th frame image, if m1 lane is matched with n1 lane, m2 lane is matched with n2 lane, m3 is matched with the remaining n3 lane, and if m3 lane is matched with n3 lane is not in the second similarity comparison result according to the second similarity comparison result, then re-matching is required, in this example, when re-matching is performed, m1 lane is matched with n2 lane, m2 lane is matched with n3 lane, m3 lane is matched with n1 lane, that is, the similarity of each lane segmentation in the i-1 th frame image and the lanes of the i-1 th frame image is matched with the i-1 th lane, and the matching of the i-1 th frame image and the lanes of the i-1 th frame image are spliced together and loaded and matched Wiring to obtain the lane segmentation matching image.
Similarly, in the matching process, if the number of each lane in the lane segmentation image of the i-1 th frame image is different from the number of each lane in the lane segmentation image of the i-th frame image, if the number of the lanes in the lane segmentation image of the i-1 th frame image is m1, m2, m3, and the number of the lanes in the lane segmentation image of the i-th frame image is n1, n2, it can be considered that one journey of the lanes in the lane segmentation image of the i-th frame image is ended, and it is only necessary to ensure the similarity comparison and matching of each lane n1, n2 in the lane segmentation image of the i-th frame image in the matching process; if the lanes in the lane segmentation image of the i-1 th frame image are m1 and m2, and the lanes in the lane segmentation image of the i-1 th frame image are n1, n2 and n3, it can be considered that there is a new lane in the lane segmentation image of the i-th frame image, and only the similarity comparison and matching between the lanes m1 and m2 in the lane segmentation image of the i-1 th frame image need to be ensured in the matching process, and the unmatched lane in the lane segmentation image of the i-th frame image can be compared and matched in the lane segmentation image of the next frame, i.e., the i +1 th frame image.
Step S14: judging whether a vehicle breaks rules or not in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; if the vehicle violation is determined, step S15 is executed.
It can be understood that if the fact that no vehicle violates the regulations exists in the ith frame of image is judged according to the vehicle segmentation matching image and the lane segmentation matching image, the violation judgment can be continuously carried out on the subsequent (i + 1) th frame of image by the same method.
In step S14, it is determined whether there is a vehicle violation in the i-th frame image according to the lane attribute of each lane in the lane segmentation matching image and by combining the vehicle matching line in the vehicle segmentation matching image, and if there is a lane with a preset lane attribute in the lane segmentation matching image and there is a vehicle matching line extending to the lane with the preset lane attribute and other lanes, it is determined that the vehicle corresponding to the vehicle matching line is a violation vehicle.
In this embodiment, the preset lane attribute includes one, two or more of a real line lane, a diversion line lane and a bus lane, when the lane attribute of the lane in the lane segmentation matching image is identified to have the preset lane attribute, whether the vehicle violates the regulations is judged by detecting whether the vehicle matching line of the vehicle extends to the lane with the preset lane attribute and other lanes, and the following description is respectively made for the real line lane, the diversion line lane and the bus lane, when the lane attribute of the lane in the lane segmentation matching image is identified to be the real line lane, if the vehicle matching line of the vehicle extends from one side lane of the real line lane to the other side of the real line lane, the vehicle can be considered to change lane across the real line, and violation is caused; when the lane attribute of the lane in the lane segmentation matching image is identified as a guide line, if the vehicle matching line of a vehicle extends from the outer lane of the guide line to the inner side of the guide line, the vehicle can be considered to occupy the guide line, and violation of regulations is caused; when the lane attribute of the lane in the lane segmentation matching image is identified as a bus lane, if the vehicle matching line of a vehicle extends from the left lane of the bus lane to the right side of the bus lane, the vehicle can be considered to occupy the bus lane, and violation of regulations is caused.
Step S15: and extracting the vehicle information of the violation vehicle according to the ith frame image.
When the vehicle violation is detected, the vehicle information of the vehicle violating the regulations can be extracted according to the ith frame image. In this embodiment, the vehicle information of the violation vehicle may be a license plate number of the violation vehicle.
Step S16: and judging whether the vehicle information of the violation vehicle is successfully extracted, if so, executing the step S17, and if not, executing the step S18.
It should be noted that when a vehicle violates a rule, for some reason, such as too far distance or obstruction, the vehicle information of the vehicle violating the rule may be blurred or hidden in the i-th frame image, and the vehicle information of the vehicle violating the rule cannot be extracted from the i-th frame image. Therefore, it is necessary to first determine whether the vehicle information extraction of the violation vehicle is successful.
Step S17: and sending the extracted vehicle information of the violation vehicle and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image for violation reporting.
If the vehicle information of the violation vehicle is successfully extracted, storing the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to violation report; and uploading the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report to a remote server at intervals of predetermined time or when a preset condition is met, and deleting the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report after the uploading is finished. In this embodiment, the predetermined time may be set to be a time when the system is idle, such as uploading a cached violation record at night; the preset condition can also be set as uploading the cached violation record when the vehicle violation is not detected at a time interval.
Step S18: acquiring the identity of the violation vehicle; and acquiring corresponding vehicle information in the cached video image according to the identity mark, and sending the corresponding vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image for violation reporting.
If the extraction of the vehicle information of the violation vehicle fails, corresponding vehicle information needs to be acquired in the cached video image through the identity of the violation vehicle. Similarly, after the vehicle information is acquired, the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report can be stored; and uploading the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report to a remote server at intervals of predetermined time or when a preset condition is met, and deleting the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report after the uploading is finished.
The real-time vehicle violation detection method of the embodiment of the invention sequentially identifies the vehicles and lane lines of the ith frame image and the (i-1) th frame image by acquiring the video image in real time and generates the identification marks for each identified vehicle, and segmenting and matching the vehicle and the lane to the ith frame image and the (i-1) th frame image according to the identification result to obtain a vehicle segmentation matching image and a lane segmentation matching image, judging whether the vehicle violates the regulations in the ith frame image according to the vehicle segmentation matching image and the lane segmentation matching image, and extracts the violation vehicle information from the violation vehicle, realizes the real-time automatic detection of the violation vehicle and the violation vehicle information extraction, has high detection efficiency without manual intervention in the whole detection process, and each frame is identified, so that missing frames cannot occur, and missing detection in the snapshot gap is avoided.
Furthermore, when the illegal vehicle information extraction fails, the corresponding vehicle information is obtained in the cached video image through the identity of the illegal vehicle, so that the accuracy and the effectiveness of the illegal vehicle information extraction are guaranteed.
And further, sequentially carrying out vehicle and lane line identification on the ith frame image and the (i-1) th frame image, and generating an identity of each vehicle for each identified vehicle according to the vehicle attribute, so that the identity of each vehicle in the video image is unique.
Furthermore, the similarity between each vehicle and each lane line in the vehicle segmentation image of the i-1 frame image and each vehicle and each lane line in the lane line segmentation image of the i-1 frame image and each lane line in the vehicle segmentation image of the i-1 frame image is compared with the similarity between the obtained vehicle segmentation matching image and the obtained lane line segmentation matching image, so that the algorithm is simple, the operation efficiency is high, and the vehicle and the lane line can be matched quickly and accurately.
Furthermore, whether the vehicle violates the regulations is judged according to the lane attributes of each lane in the lane segmentation matching image and by combining the vehicle matching line in the vehicle segmentation matching image, the automatic detection of the vehicle violations can be completed through one frame of image or two continuous frames of images, the detection efficiency is high, and the detection result is accurate.
Further, when the video image exceeds the storage capacity, at least part of the video image which is stored firstly is deleted, the real-time violation detection result is not influenced, and the storage space can be released.
Further, vehicle information related to violation reporting and the ith frame image or the ith frame image and the (i-1) th frame image are stored, and violation evidence can be effectively stored.
Further, the stored vehicle information related to violation reporting and the ith frame image or the ith frame image and the (i-1) th frame image are stored and uploaded to a remote server every predetermined time or when preset conditions are met, local storage is deleted after uploading is completed, violation evidence can be uploaded when the system is idle, uploading success rate is guaranteed, the local storage is deleted after uploading is successful, and local storage space is further released.
Fig. 3 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the invention. As shown in fig. 3, the real-time vehicle violation detecting device 20 includes a video receiving module 21, a video caching module 22, an information extracting module 23, a violation judging module 24, a violation information extracting module 25, a violation information storing module 26, and a violation information sending module 27.
The video receiving module 21 is used for acquiring a real-time video image; in this embodiment, the video receiving module 21 may receive a video image shot by the vehicle-mounted camera during driving.
A video caching module 22, configured to cache the video image obtained by the video receiving module 21; at least a portion of the video image that was first stored may be deleted when the video image exceeds the storage capacity of the video cache module 22.
An information extraction module 23, configured to perform vehicle and lane line recognition on an i-th frame image and an i-1 th frame image in the video image acquired by the video receiving module 21, generate an identity for each recognized vehicle, and perform vehicle and lane segmentation and matching on the i-th frame image and the i-1 th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, where i is a natural number greater than or equal to 2, where the information extraction module 23 may respectively recognize a vehicle and a lane line as a target object by using an image recognition technology for each frame image, and acquire each lane and its type, each vehicle and its vehicle attribute in each frame image by recognition, in this embodiment, the vehicle attribute includes one, two, or more of a color, a vehicle type, a brand, and a license plate, the lane type comprises one, two or more of a solid line lane, a flow guide line lane and a bus lane. And generating the identity of each vehicle according to the vehicle attributes.
Through an image recognition technology, the information extraction module 23 may respectively recognize the vehicle and the lane line in each frame of image, segment the vehicle and the lane line from the image, and respectively obtain the vehicle segmentation image and the lane segmentation image.
Specifically, similarity comparison is performed between each vehicle in the vehicle segmentation image of the i-1 th frame image and each vehicle in the vehicle segmentation image of the i-th frame image to obtain a first similarity comparison result, and the first similarity comparison result indicates that each vehicle including the i-1 th frame image and at least one vehicle of the i-th frame image are similar vehicles; randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the vehicle in the i-th frame image or performing residual matching with the unmatched vehicle in the i-th frame image;
judging whether the residual matching result is reasonable according to the first similarity ratio, if so, splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together and loading a matched line on the matched vehicle to obtain the vehicle segmentation matched image, if not, matching the vehicle of the ith-1 frame image with the other similar vehicle corresponding to the vehicle in the ith frame image, if so, splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together and loading the matched line on the matched vehicle to obtain the vehicle segmentation matched image.
The information extraction module 23 is configured to perform similarity comparison between the lane segmentation image of the i-1 th frame image and the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, where the second similarity comparison result includes that each lane of the i-1 th frame image and at least one lane of the i-th frame image are similar lanes; randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the lane in the i-th frame image or performing residual matching with the unmatched lane in the i-th frame image,
judging whether the residual matching result is reasonable or not according to the second similarity comparison result; if the lane segmentation matching image is not reasonable, the lane of the i-1 frame image is matched with the other similar lane corresponding to the lane in the i-1 frame image, until the remaining matching result is judged to be reasonable, the lane segmentation images of the i-1 frame image and the lane segmentation images of the i-1 frame image are spliced together, the matched lane is loaded with the matched line, and the lane segmentation matching image is obtained.
The violation judging module 24 is configured to judge whether a vehicle violates a rule or not in the i-th frame image according to the vehicle segmentation matching image extracted by the information extracting module 23 and the lane segmentation matching image; and when a vehicle violation is judged, extracting vehicle information of the vehicle violating the regulations according to the i-th frame image, specifically, judging whether the vehicle violating the regulations exists in the i-th frame image by the violation judging module 24 according to the lane attributes of all lanes in the lane segmentation matching image and combining the vehicle matching line in the vehicle segmentation matching image to judge whether the vehicle violating the regulations exists, and if the lane segmentation matching image has lanes with preset lane attributes and the vehicle matching line extends to the lanes with the preset lane attributes and other lanes, judging that the vehicle corresponding to the vehicle matching line is the vehicle violating the regulations. When the vehicle violation is detected, the vehicle information of the vehicle violating the regulations can be extracted according to the ith frame image. In this embodiment, the vehicle information of the violation vehicle may be a license plate number of the vehicle.
If the violation judging module 24 successfully extracts the vehicle information of the violation vehicle, the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to violation reporting are sent to the violation information storage module 26 for storage; and a violation information sending module 27 uploads the stored vehicle information and the frame i image or the vehicle information and the frame i image and the frame i-1 image which are related to violation report to a remote server from the violation information storage module 26 every predetermined time or when a preset condition is met, and deletes the vehicle information and the frame i image or the vehicle information and the frame i image and the frame i-1 image which are related to violation report and stored in the violation information storage module 26 after the uploading is finished.
If the violation judging module 24 fails to extract the violation vehicle information, the identity of the violation vehicle is sent to the violation information extracting module 25, and the violation information extracting module 25 obtains corresponding vehicle information from the video image cached in the video receiving module 21 according to the identity of the violation vehicle, where the vehicle information may be a license plate number in this embodiment. Similarly, after the violation information extraction module 25 obtains the vehicle information, the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report may be sent to the violation information storage module 26 for storage; and a violation information sending module 27 uploads the stored vehicle information and the frame i image or the vehicle information and the frame i image and the frame i-1 image which are related to violation report to a remote server from the violation information storage module 26 every predetermined time or when a preset condition is met, and deletes the vehicle information and the frame i image or the vehicle information and the frame i image and the frame i-1 image which are related to violation report and stored in the violation information storage module 26 after the uploading is finished.
It can be understood that the specific manner of implementing each function by each module of the real-time vehicle violation detecting device may refer to the specific steps corresponding to the above embodiment, and therefore, no further description is provided herein.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the invention. As shown in fig. 4, the real-time vehicle violation detection device 30 includes a memory 32, a processor 31, and a computer program stored on the memory 32 and operable on the processor 31, the processor 31 when executing the computer program implementing the steps of: acquiring a real-time video image; caching the video image; recognizing vehicles and lane lines of an ith frame image and an i-1 th frame image in the video image, generating identification marks for each recognized vehicle, and segmenting and matching the vehicles and lanes of the ith frame image and the i-1 th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2; judging whether a vehicle breaks rules or not in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; if the vehicle violation is judged, extracting violation vehicle information according to the ith frame image; if the illegal vehicle information is successfully extracted, sending the extracted illegal vehicle information and the ith frame image or the ith frame image and the (i-1) th frame image for illegal reporting; if the illegal vehicle information extraction fails, acquiring the identity of the illegal vehicle; and acquiring corresponding vehicle information in the cached video image according to the identity mark, and sending the corresponding vehicle information and the ith frame image or the corresponding vehicle information and the ith frame image and the (i-1) th frame image for violation reporting.
In one embodiment, the step of recognizing the vehicle and lane lines of the i-th frame image and the i-1 th frame image in the video image, generating an identity for each recognized vehicle, and segmenting and matching the vehicle and lane of the i-th frame image and the i-1 th frame image according to the recognition result to obtain the vehicle segmentation matching image and the lane segmentation matching image includes: recognizing each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of solid line lane, guide line lane and bus lane; generating an identity of each vehicle according to the vehicle attribute; dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image; carrying out vehicle matching on the ith frame image and the vehicle segmentation image of the (i-1) th frame image to obtain the vehicle segmentation matching image; and carrying out lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image.
In one embodiment, the step of performing vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1 th frame image to obtain the vehicle segmentation matching image comprises: comparing the similarity of each vehicle in the vehicle segmentation image of the i-1 frame image with each vehicle in the vehicle segmentation image of the i frame image to obtain a first similarity comparison result, wherein the first similarity comparison result indicates that each vehicle comprising the i-1 frame image and at least one vehicle of the i frame image are similar vehicles; randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the vehicle in the i-th frame image or performing residual matching with the unmatched vehicle in the i-th frame image; judging whether the residual matching result is reasonable according to the first similarity ratio, if so, splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together and loading a matched line on the matched vehicle to obtain the vehicle segmentation matched image, if not, matching the vehicle of the ith-1 frame image with the other similar vehicle corresponding to the vehicle in the ith frame image, if so, splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together and loading the matched line on the matched vehicle to obtain the vehicle segmentation matched image.
In one embodiment, the step of performing lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image comprises: carrying out similarity comparison on each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that each lane of the i-1 th frame image and at least one lane of the i-1 th frame image are similar lanes; randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the lane in the i-1 th frame image or performing residual matching with the unmatched lane in the i-th frame image, and judging whether the residual matching result is reasonable or not according to the second similarity comparison result; if the lane segmentation matching image is not reasonable, the lane of the i-1 frame image is matched with the other similar lane corresponding to the lane in the i-1 frame image, until the remaining matching result is judged to be reasonable, the lane segmentation images of the i-1 frame image and the lane segmentation images of the i-1 frame image are spliced together, the matched lane is loaded with the matched line, and the lane segmentation matching image is obtained.
In one embodiment, the step of judging whether a vehicle violation exists in the ith frame image according to the vehicle segmentation matching image and the lane segmentation matching image comprises the following steps: and judging whether a vehicle violates a regulation or not according to the lane attributes of each lane in the lane segmentation matching image and by combining a vehicle matching line in the vehicle segmentation matching image, and if the lane with the preset lane attribute in the lane segmentation matching image and the vehicle matching line extend to the lane with the preset lane attribute and other lanes, judging that the vehicle corresponding to the vehicle matching line is a violation vehicle.
In one embodiment, the step of buffering the video image comprises: deleting at least a portion of the video image that was first stored when the video image exceeds storage capacity; the at least part of the video image is a frame of video image.
In one embodiment, the processor 31, when executing the computer readable instructions, further performs the steps of: storing the vehicle information and the ith frame image related to the violation report or the corresponding vehicle information and the ith frame image and the (i-1) th frame image; and uploading the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the (i-1) th frame image to a remote server at intervals of predetermined time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the (i-1) th frame image after the uploading is finished.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a storage medium according to an embodiment of the invention. As shown in fig. 5, a storage medium storing computer readable instructions 41, the computer readable instructions 41, when executed by one or more processors, cause the one or more processors to perform the steps of: acquiring a real-time video image; caching the video image; recognizing vehicles and lane lines of an ith frame image and an i-1 th frame image in the video image, generating identification marks for each recognized vehicle, and segmenting and matching the vehicles and lanes of the ith frame image and the i-1 th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2; judging whether a vehicle breaks rules or not in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; if the vehicle violation is judged, extracting violation vehicle information according to the ith frame image; if the illegal vehicle information is successfully extracted, sending the extracted illegal vehicle information and the ith frame image or the ith frame image and the (i-1) th frame image for illegal reporting; if the illegal vehicle information extraction fails, acquiring the identity of the illegal vehicle; and acquiring corresponding vehicle information in the cached video image according to the identity mark, and sending the corresponding vehicle information and the ith frame image or the corresponding vehicle information and the ith frame image and the (i-1) th frame image for violation reporting.
In one embodiment, the step of recognizing the vehicle and lane lines of the i-th frame image and the i-1 th frame image in the video image, generating an identity for each recognized vehicle, and segmenting and matching the vehicle and lane of the i-th frame image and the i-1 th frame image according to the recognition result to obtain the vehicle segmentation matching image and the lane segmentation matching image includes: recognizing each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of solid line lane, guide line lane and bus lane; generating an identity of each vehicle according to the vehicle attribute; dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image; carrying out vehicle matching on the ith frame image and the vehicle segmentation image of the (i-1) th frame image to obtain the vehicle segmentation matching image; and carrying out lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image.
In one embodiment, the step of performing vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1 th frame image to obtain the vehicle segmentation matching image comprises: comparing the similarity of each vehicle in the vehicle segmentation image of the i-1 frame image with each vehicle in the vehicle segmentation image of the i frame image to obtain a first similarity comparison result, wherein the first similarity comparison result indicates that each vehicle comprising the i-1 frame image and at least one vehicle of the i frame image are similar vehicles; randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the vehicle in the i-th frame image or performing residual matching with the unmatched vehicle in the i-th frame image; judging whether the residual matching result is reasonable according to the first similarity ratio, if so, splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together and loading a matched line on the matched vehicle to obtain the vehicle segmentation matched image, if not, matching the vehicle of the ith-1 frame image with the other similar vehicle corresponding to the vehicle in the ith frame image, if so, splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together and loading the matched line on the matched vehicle to obtain the vehicle segmentation matched image.
In one embodiment, the step of performing lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image comprises: carrying out similarity comparison on each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that each lane of the i-1 th frame image and at least one lane of the i-1 th frame image are similar lanes; randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the lane in the i-1 th frame image or performing residual matching with the unmatched lane in the i-th frame image, and judging whether the residual matching result is reasonable or not according to the second similarity comparison result; if the lane segmentation matching image is not reasonable, the lane of the i-1 frame image is matched with the other similar lane corresponding to the lane in the i-1 frame image, until the remaining matching result is judged to be reasonable, the lane segmentation images of the i-1 frame image and the lane segmentation images of the i-1 frame image are spliced together, the matched lane is loaded with the matched line, and the lane segmentation matching image is obtained.
In one embodiment, the step of judging whether a vehicle violation exists in the ith frame image according to the vehicle segmentation matching image and the lane segmentation matching image comprises the following steps: and judging whether a vehicle violates a regulation or not according to the lane attributes of each lane in the lane segmentation matching image and by combining a vehicle matching line in the vehicle segmentation matching image, and if the lane with the preset lane attribute in the lane segmentation matching image and the vehicle matching line extend to the lane with the preset lane attribute and other lanes, judging that the vehicle corresponding to the vehicle matching line is a violation vehicle.
In one embodiment, the step of buffering the video image comprises: deleting at least a portion of the video image that was first stored when the video image exceeds storage capacity; the at least part of the video image is a frame of video image.
In one embodiment, the processor executing the computer readable instructions 41 further performs the steps of: storing the vehicle information and the ith frame image related to the violation report or the corresponding vehicle information and the ith frame image and the (i-1) th frame image; and uploading the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the (i-1) th frame image to a remote server at intervals of predetermined time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the (i-1) th frame image after the uploading is finished.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the computer program is executed. The storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a Random Access Memory (RAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A real-time vehicle violation detection method is characterized by comprising the following steps:
acquiring a real-time video image;
caching the video image;
recognizing vehicles and lane lines of an ith frame image and an i-1 th frame image in the video image, generating identification marks for each recognized vehicle, and segmenting and matching the vehicles and lanes of the ith frame image and the i-1 th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2;
judging whether a vehicle breaks rules or not in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image;
if the vehicle violation is judged, extracting the vehicle information of the vehicle violating the regulations according to the ith frame image;
if the vehicle information is successfully extracted, the extracted vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image are sent for violation reporting;
if the vehicle information extraction fails, acquiring the identity of the violation vehicle; and
and acquiring corresponding vehicle information in the cached video image according to the identity mark, and sending the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image for violation reporting.
2. The real-time vehicle violation detecting method of claim 1, wherein the identifying the vehicle and lane lines of the i-th frame image and the i-1 th frame image in the video image, generating an identity for each identified vehicle, and segmenting and matching the vehicle and lane of the i-th frame image and the i-1 th frame image according to the identification result to obtain the vehicle segmentation matching image and the lane segmentation matching image comprises:
recognizing each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of solid line lane, guide line lane and bus lane;
generating an identity of each vehicle according to the vehicle attribute;
dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image;
carrying out vehicle matching on the ith frame image and the vehicle segmentation image of the (i-1) th frame image to obtain the vehicle segmentation matching image; and
and carrying out lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image.
3. The real-time vehicle violation detection method of claim 2 wherein the step of performing vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1 th frame image to obtain the vehicle segmentation matching image comprises:
comparing the similarity of each vehicle in the vehicle segmentation image of the i-1 frame image with each vehicle in the vehicle segmentation image of the i frame image to obtain a first similarity comparison result, wherein the first similarity comparison result indicates that each vehicle comprising the i-1 frame image and at least one vehicle of the i frame image are similar vehicles;
randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the vehicle in the i-th frame image or performing residual matching with the unmatched vehicle in the i-th frame image;
judging whether the residual matching result is reasonable according to the first similarity ratio pair result,
if the image is reasonable, splicing the vehicle segmentation images of the ith frame image and the (i-1) th frame image together and loading a matched vehicle on a matched line so as to obtain the vehicle segmentation matched image,
and if the vehicle is not reasonable, matching the vehicle of the i-1 th frame image with the other corresponding similar vehicle in the i-1 th frame image until the residual matching result is judged to be reasonable, splicing the vehicle segmentation images of the i-1 th frame image and the i-1 th frame image together, and loading a matched line on the matched vehicle so as to obtain the vehicle segmentation matching image.
4. The real-time vehicle violation detection method of claim 2 wherein performing lane matching on the i-th frame image and the lane segmentation image of the i-1 th frame image to obtain the lane segmentation matching image comprises:
carrying out similarity comparison on each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that each lane of the i-1 th frame image and at least one lane of the i-1 th frame image are similar lanes;
randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the lane in the i-th frame image or performing residual matching with the unmatched lane in the i-th frame image,
judging whether the residual matching result is reasonable or not according to the second similarity comparison result;
if the lane segmentation images are reasonable, splicing the i-th frame image and the lane segmentation images of the i-1 th frame image together and loading matched lines on matched lanes to obtain the lane segmentation matched images,
and if the lane is not reasonable, matching the lane of the i-1 th frame image with the other corresponding similar lane in the i-1 th frame image until the residual matching result is judged to be reasonable, splicing the i-th frame image and the lane segmentation image of the i-1 th frame image together, and loading a matched line on the matched lane so as to obtain the lane segmentation matching image.
5. The real-time vehicle violation detecting method of claim 2 wherein said determining whether a vehicle violation exists in said ith frame of image based on said vehicle segmentation matching image and said lane segmentation matching image comprises:
and judging whether a vehicle violates a regulation or not according to the lane attributes of each lane in the lane segmentation matching image and by combining a vehicle matching line in the vehicle segmentation matching image, and if the lane with the preset lane attribute in the lane segmentation matching image and the vehicle matching line extend to the lane with the preset lane attribute and other lanes, judging that the vehicle corresponding to the vehicle matching line is a violation vehicle.
6. The real-time vehicle violation detection method of claim 1 wherein caching the video image further comprises:
deleting at least a portion of the video image that was first stored when the video image exceeds storage capacity; the at least part of the video image is a frame of video image.
7. The real-time vehicle violation detection method of claim 1 wherein said real-time vehicle violation detection method further comprises:
storing the vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report; and
and uploading the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report to a remote server at preset intervals or when preset conditions are met, and deleting the stored vehicle information and the ith frame image or the vehicle information and the ith frame image and the (i-1) th frame image related to the violation report after the uploading is finished.
8. A real-time vehicle violation detection device, comprising:
the video receiving module is used for acquiring a real-time video image;
the video caching module is used for caching the video image;
the information extraction module is used for recognizing vehicles and lane lines of the ith frame image and the (i-1) th frame image in the video image, generating an identity for each recognized vehicle, and segmenting and matching the vehicles and lanes of the ith frame image and the (i-1) th frame image according to a recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number more than or equal to 2;
the violation judging module is used for judging whether a vehicle violates the regulations in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; when the fact that the vehicle breaks rules is judged, vehicle information of the vehicle breaking rules is extracted according to the ith frame image;
the violation information extraction module is used for acquiring the identity of the violation vehicle when the violation judgment module fails to extract the vehicle information of the violation vehicle; acquiring corresponding vehicle information in the cached video image according to the identity;
the violation information storage module is used for storing the vehicle information and the ith frame image or the vehicle information, the ith frame image and the (i-1) th frame image which are related to violation report;
and the violation information sending module is used for sending the vehicle information related to violation report and the ith frame image or the vehicle information, the ith frame image and the (i-1) th frame image to carry out violation report.
9. A real-time vehicle violation detection device comprising a processor, a memory coupled to the processor, wherein,
the memory has stored therein computer readable instructions which, when executed by the processor, cause the processor to carry out the steps of the real-time vehicle violation detection method of any one of claims 1-7.
10. A storage medium having stored thereon computer readable instructions which, when executed by one or more processors, cause the one or more processors to carry out the steps of the real-time vehicle violation detection method as claimed in any one of claims 1-7.
CN202010470098.2A 2020-05-28 2020-05-28 Real-time vehicle violation detection method, device, equipment and storage medium Active CN111666853B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010470098.2A CN111666853B (en) 2020-05-28 2020-05-28 Real-time vehicle violation detection method, device, equipment and storage medium
PCT/CN2020/118438 WO2021120776A1 (en) 2020-05-28 2020-09-28 Real-time vehicle violation detection method, apparatus, device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010470098.2A CN111666853B (en) 2020-05-28 2020-05-28 Real-time vehicle violation detection method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111666853A true CN111666853A (en) 2020-09-15
CN111666853B CN111666853B (en) 2023-06-02

Family

ID=72385108

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010470098.2A Active CN111666853B (en) 2020-05-28 2020-05-28 Real-time vehicle violation detection method, device, equipment and storage medium

Country Status (2)

Country Link
CN (1) CN111666853B (en)
WO (1) WO2021120776A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112348992A (en) * 2020-11-12 2021-02-09 广东利通科技投资有限公司 Vehicle-mounted video processing method and device based on vehicle-road cooperative system and storage medium
CN112712703A (en) * 2020-12-09 2021-04-27 上海眼控科技股份有限公司 Vehicle video processing method and device, computer equipment and storage medium
CN112712708A (en) * 2020-12-28 2021-04-27 上海眼控科技股份有限公司 Information detection method, device, equipment and storage medium
WO2021120776A1 (en) * 2020-05-28 2021-06-24 平安科技（深圳）有限公司 Real-time vehicle violation detection method, apparatus, device, and storage medium
CN114387533A (en) * 2022-01-07 2022-04-22 北京远度互联科技有限公司 Method and device for identifying road violation, electronic equipment and storage medium
CN115206091A (en) * 2022-06-07 2022-10-18 西安电子科技大学广州研究院 Road condition and event monitoring system and method based on multiple cameras and millimeter wave radar
CN115359665A (en) * 2022-08-17 2022-11-18 北京博宏科元信息科技有限公司 Multichannel violation vehicle recording method and device based on radio frequency video all-in-one machine

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113450578B (en) * 2021-06-25 2022-08-12 北京市商汤科技开发有限公司 Traffic violation event evidence obtaining method, device, equipment and system
CN113642412B (en) * 2021-07-16 2023-12-26 盛视科技股份有限公司 Method, device and equipment for detecting vehicle occupying bus lane

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105185114A (en) * 2015-09-07 2015-12-23 北京信路威科技股份有限公司 Device and method for collecting evidence of vehicle regulation-violating lane changing behavior
CN106652468A (en) * 2016-12-09 2017-05-10 武汉极目智能技术有限公司 Device and method for detection of violation of front vehicle and early warning of violation of vehicle on road
CN106652465A (en) * 2016-11-15 2017-05-10 成都通甲优博科技有限责任公司 Method and system for identifying abnormal driving behavior on road
CN107481526A (en) * 2017-09-07 2017-12-15 公安部第三研究所 System and method for drive a vehicle lane change detection record and lane change violating the regulations report control
CN110706261A (en) * 2019-10-22 2020-01-17 上海眼控科技股份有限公司 Vehicle violation detection method and device, computer equipment and storage medium
US20200117928A1 (en) * 2018-10-12 2020-04-16 Toyota Jidosha Kabushiki Kaisha Traffic violation vehicle identification system, server and non-transitory recording medium in which vehicle control program is recorded

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105809088B (en) * 2014-12-30 2019-07-19 清华大学 Vehicle identification method and system
CN106991820B (en) * 2016-01-20 2021-03-23 中兴通讯股份有限公司 Illegal vehicle processing method and device
US10740607B2 (en) * 2017-08-18 2020-08-11 Autel Robotics Co., Ltd. Method for determining target through intelligent following of unmanned aerial vehicle, unmanned aerial vehicle and remote control
CN109740550A (en) * 2019-01-08 2019-05-10 哈尔滨理工大学 A kind of lane detection and tracking method based on monocular vision
CN110610153A (en) * 2019-09-10 2019-12-24 重庆工程职业技术学院 Lane recognition method and system for automatic driving
CN111666853B (en) * 2020-05-28 2023-06-02 平安科技（深圳）有限公司 Real-time vehicle violation detection method, device, equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105185114A (en) * 2015-09-07 2015-12-23 北京信路威科技股份有限公司 Device and method for collecting evidence of vehicle regulation-violating lane changing behavior
CN106652465A (en) * 2016-11-15 2017-05-10 成都通甲优博科技有限责任公司 Method and system for identifying abnormal driving behavior on road
CN106652468A (en) * 2016-12-09 2017-05-10 武汉极目智能技术有限公司 Device and method for detection of violation of front vehicle and early warning of violation of vehicle on road
CN107481526A (en) * 2017-09-07 2017-12-15 公安部第三研究所 System and method for drive a vehicle lane change detection record and lane change violating the regulations report control
US20200117928A1 (en) * 2018-10-12 2020-04-16 Toyota Jidosha Kabushiki Kaisha Traffic violation vehicle identification system, server and non-transitory recording medium in which vehicle control program is recorded
CN110706261A (en) * 2019-10-22 2020-01-17 上海眼控科技股份有限公司 Vehicle violation detection method and device, computer equipment and storage medium

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021120776A1 (en) * 2020-05-28 2021-06-24 平安科技（深圳）有限公司 Real-time vehicle violation detection method, apparatus, device, and storage medium
CN112348992A (en) * 2020-11-12 2021-02-09 广东利通科技投资有限公司 Vehicle-mounted video processing method and device based on vehicle-road cooperative system and storage medium
CN112712703A (en) * 2020-12-09 2021-04-27 上海眼控科技股份有限公司 Vehicle video processing method and device, computer equipment and storage medium
CN112712708A (en) * 2020-12-28 2021-04-27 上海眼控科技股份有限公司 Information detection method, device, equipment and storage medium
CN114387533A (en) * 2022-01-07 2022-04-22 北京远度互联科技有限公司 Method and device for identifying road violation, electronic equipment and storage medium
CN115206091A (en) * 2022-06-07 2022-10-18 西安电子科技大学广州研究院 Road condition and event monitoring system and method based on multiple cameras and millimeter wave radar
CN115359665A (en) * 2022-08-17 2022-11-18 北京博宏科元信息科技有限公司 Multichannel violation vehicle recording method and device based on radio frequency video all-in-one machine

Also Published As

Publication number Publication date
CN111666853B (en) 2023-06-02
WO2021120776A1 (en) 2021-06-24

Similar Documents

Publication Publication Date Title
CN111666853B (en) Real-time vehicle violation detection method, device, equipment and storage medium
CN107967806B (en) Vehicle fake-license detection method, device, readable storage medium storing program for executing and electronic equipment
JP7218535B2 (en) Traffic violation vehicle identification system and server
US10212397B2 (en) Abandoned object detection apparatus and method and system
CN110532916B (en) Motion trail determination method and device
CN108932851B (en) Method and device for recording illegal parking behaviors of motor vehicle
CN110580808B (en) Information processing method and device, electronic equipment and intelligent traffic system
CN108932849B (en) Method and device for recording low-speed running illegal behaviors of multiple motor vehicles
CN110032947B (en) Method and device for monitoring occurrence of event
CN108932852B (en) Method and device for recording behaviors of illegal occupation of emergency lane by motor vehicle
CN111898581A (en) Animal detection method, device, electronic equipment and readable storage medium
CN114037924A (en) Vehicle brake-passing judgment method based on image recognition technology and related device
US11182983B2 (en) Same vehicle detection device, toll collection facility, same vehicle detection method, and program
CN111292530A (en) Method, device, server and storage medium for processing violation pictures
CN113781827A (en) Video data management method of cloud platform and cloud platform
CN109003457B (en) Method and device for recording behaviors of multiple motor vehicles illegally occupying emergency lane
CN112084905B (en) Traffic light state identification method, system, equipment and storage medium
CN109300313B (en) Illegal behavior detection method, camera and server
CN111383248A (en) Method and device for judging red light running of pedestrian and electronic equipment
JP2002190013A (en) System and method for detecting congestion by image recognition
CN110660225A (en) Red light running behavior detection method, device and equipment
CN108922175B (en) Method and device for recording illegal behaviors of multiple motor vehicles crossing solid line
CN109003456B (en) Method and system for recording illegal actions of crossing solid line of motor vehicle
JP7434394B2 (en) Method for determining images likely to have false negative object detection
CN116363628A (en) Mark detection method and device, nonvolatile storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40030808

Country of ref document: HK

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant