CN111666853B

CN111666853B - Real-time vehicle violation detection method, device, equipment and storage medium

Info

Publication number: CN111666853B
Application number: CN202010470098.2A
Authority: CN
Inventors: 芦文峰; 刘伟超; 郭倜颖; 曾凡涛; 陈远旭
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2020-05-28
Filing date: 2020-05-28
Publication date: 2023-06-02
Anticipated expiration: 2040-05-28
Also published as: WO2021120776A1; CN111666853A

Abstract

The invention provides a real-time vehicle violation detection method, device, equipment and storage medium. The method comprises the following steps: acquiring and caching real-time video images; identifying vehicle and lane lines of an ith frame image and an ith-1 frame image in the video image, generating identification marks for the identified vehicles, and carrying out vehicle and lane segmentation and matching on the ith frame image and the ith-1 frame image according to identification results to obtain vehicle segmentation matching images and lane segmentation matching images; judging whether a violation exists or not according to the vehicle segmentation matching image and the lane segmentation matching image; if yes, extracting vehicle information and reporting violations according to the ith frame image; if the vehicle information extraction fails, acquiring the identity of the illegal vehicle, and acquiring the vehicle information and reporting the illegal vehicle in the video image. The intelligent traffic system and the intelligent traffic system can achieve real-time automatic violation detection, and can be applied to intelligent traffic scenes, so that the aim of building intelligent cities is promoted.

Description

Real-time vehicle violation detection method, device, equipment and storage medium

Technical Field

The present invention relates to the field of video processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting a real-time vehicle violation.

Background

The intelligent detection of traffic violations uses various sensors and image acquisition equipment, and combines a rear-end violation judgment algorithm and an information extraction algorithm to automatically judge the violations of the vehicles. The technology can help traffic management departments to reduce labor cost and avoid errors such as false detection, omission detection and the like caused by human reasons while improving the speed and accuracy of illegal judgment.

At present, the intelligent violation detection technology on the market is divided into two types: a hardware device for detecting violations is installed at a fixed position, a sensor such as light or pressure is used for sensing information such as the position of a vehicle, and then a single photo or a plurality of photos obtained through photographing are combined to judge violations and extract information. The hardware device for detecting the violations can only be installed at a fixed position to detect specific violations, is not flexible enough and can only monitor the vehicle at the fixed position.

The other is to acquire driving video through a vehicle-mounted camera, and then process one or a plurality of frames in the video locally or send the video to a remote server for processing to judge rule violations. Processing one or a plurality of frames in the video locally may cause difficulty in extracting vehicle information, such as a license plate in an extracted image being blocked; sending to a remote server for processing places a burden on server downloads and cannot be used on a large scale.

Disclosure of Invention

The invention provides a real-time vehicle violation detection method, device, equipment and storage device, which can achieve the purposes of real-time automatic violation detection and accurate extraction of violation vehicle information.

In order to solve the technical problems, the invention adopts a technical scheme that: the real-time vehicle violation detection method comprises the following steps:

acquiring a real-time video image;

caching the video image;

identifying vehicle and lane lines of an ith frame image and an ith-1 frame image in the video image, generating identification marks for each identified vehicle, and dividing and matching the ith frame image and the ith-1 frame image according to identification results to obtain a vehicle division matching image and a lane division matching image, wherein i is a natural number greater than or equal to 2;

judging whether a vehicle violation exists in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image;

if the vehicle violations are judged, vehicle information of the violating vehicles is extracted according to the ith frame image;

if the vehicle information is successfully extracted, the extracted vehicle information and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image are sent to report in a violation manner;

If the vehicle information extraction fails, acquiring the identity of the illegal vehicle; a kind of electronic device with high-pressure air-conditioning system

And acquiring corresponding vehicle information from the cached video images according to the identity, and transmitting the vehicle information and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image for illegal reporting.

According to an embodiment of the present invention, the identifying the vehicle and the lane line of the i-th frame image and the i-1-th frame image in the video image, generating an identification for each identified vehicle, and dividing and matching the vehicle and the lane of the i-th frame image and the i-1-th frame image according to the identification result, and obtaining a vehicle division matching image and a lane division matching image includes:

identifying each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of real line road, diversion line road and bus lane;

generating an identity of each vehicle according to the vehicle attribute;

dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image;

Performing vehicle matching on the vehicle segmentation image of the ith frame image and the i-1 th frame image to obtain a vehicle segmentation matching image; a kind of electronic device with high-pressure air-conditioning system

And carrying out lane matching on the lane segmentation image of the ith frame image and the i-1 th frame image to obtain the lane segmentation matching image.

According to an embodiment of the present invention, performing vehicle matching on the i-th frame image and the vehicle division image of the i-1-th frame image to obtain the vehicle division matching image includes:

performing similarity comparison on each vehicle in the vehicle segmentation image of the i-1 frame image and each vehicle in the vehicle segmentation image of the i frame image to obtain a first similarity comparison result, wherein each vehicle in the i-1 frame image and at least one vehicle in the i frame image are similar vehicles;

randomly matching each vehicle of the i-1 th frame image with one of the similar vehicles corresponding to the i-th frame image or performing residual matching with the unmatched vehicle of the i-th frame image,

judging whether the residual matching result is reasonable or not according to the first similarity comparison result;

if so, stitching the ith frame image and the vehicle segmentation image of the ith-1 frame image together and loading the matched vehicle into a matching line, thereby obtaining the vehicle segmentation matching image,

If not, matching the vehicle of the i-1 frame image with the other similar vehicles corresponding to the i frame image until the residual matching result is judged to be reasonable, splicing the i frame image and the vehicle segmentation image of the i-1 frame image together, and loading the matched vehicle into a matching line, so that the vehicle segmentation matching image is obtained.

According to an embodiment of the present invention, performing lane matching on the lane-divided images of the i-th frame image and the i-1-th frame image to obtain the lane-divided matching image includes:

carrying out similarity comparison on each lane in the lane segmentation image of the i-1 frame image and each lane in the lane segmentation image of the i frame image to obtain a second similarity comparison result, wherein each lane of the i-1 frame image and at least one lane of the i frame image are similar lanes;

randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the i-1 th frame image or carrying out residual matching with the unmatched lanes in the i-1 th frame image,

Judging whether the residual matching result is reasonable or not according to the second similarity comparison result

If so, splicing the i-th frame image and the lane segmentation image of the i-1 th frame image together and loading the matched lanes into a matching line so as to obtain the lane segmentation matching image,

if not, matching the lane of the ith-1 frame image with the other similar lanes corresponding to the ith frame image until the rest matching result is judged to be reasonable, splicing the lane segmentation images of the ith frame image and the ith-1 frame image together, and loading the matched lanes on a matching line, so that the lane segmentation matching image is obtained.

According to an embodiment of the present invention, the determining whether there is a vehicle violation in the ith frame image according to the vehicle segmentation matching image and the lane segmentation matching image includes:

judging whether a vehicle violates regulations or not according to the lane attribute of each lane in the lane segmentation matching image and combining the vehicle matching line in the vehicle segmentation matching image, and judging that the vehicle corresponding to the vehicle matching line is a violation vehicle if the lane with the preset lane attribute in the lane segmentation matching image and the vehicle matching line extends to the lane with the preset lane attribute and other lanes.

According to an embodiment of the present invention, the buffering the video image further includes:

deleting at least part of the video images stored first when the video images exceed a storage capacity; the at least part of the video image is a frame of video image.

According to one embodiment of the invention, the method further comprises: storing the vehicle information related to the violation report or the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image; and uploading the stored vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image to a remote server every preset time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image after the uploading is completed.

In addition, in order to solve the technical problems, the invention adopts a technical scheme that: there is provided a real-time vehicle violation detection device, the device comprising:

the video receiving module is used for acquiring real-time video images;

The video caching module is used for caching the video image;

the information extraction module is used for identifying vehicles and lane lines of an ith frame image and an ith-1 frame image in the video image, generating identification marks for the identified vehicles, and dividing and matching the vehicles and the lanes of the ith frame image and the ith-1 frame image according to the identification result to obtain vehicle division matching images and lane division matching images, wherein i is a natural number greater than or equal to 2;

the violation judging module is used for judging whether the ith frame image has a vehicle violation or not according to the vehicle segmentation matching image and the lane segmentation matching image; when the vehicle violations are judged, vehicle information of the violating vehicles is extracted according to the ith frame image;

the violation information extraction module is used for acquiring the identity of the violation vehicle when the vehicle information of the violation vehicle is extracted by the violation judgment module fails to be extracted; acquiring corresponding vehicle information from the cached video image according to the identity;

the violation information storage module is used for storing the vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image;

And the violation information sending module is used for sending the vehicle information related to the violation reporting and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image to report the violation.

In addition, in order to solve the technical problems, the invention adopts a technical scheme that: there is provided a real-time vehicle violation detection device comprising a processor, a memory coupled to the processor, wherein the memory has stored therein computer readable instructions that, when executed by the processor, cause the processor to perform the steps of the real-time vehicle violation detection method described above.

In addition, in order to solve the technical problems, the invention adopts a technical scheme that: there is provided a storage medium storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the real-time vehicle violation detection method described above.

According to the real-time vehicle violation detection method, device, equipment and storage device, through acquiring the video image in real time, sequentially carrying out vehicle and lane line identification on the ith frame image and the ith-1 frame image, generating identity marks for each identified vehicle, and carrying out vehicle and lane segmentation and matching on the ith frame image and the ith-1 frame image according to the identification result, obtaining a vehicle segmentation matching image and a lane segmentation matching image, judging whether a vehicle is in the ith frame image or not according to the vehicle segmentation matching image and the lane segmentation matching image, and extracting violation vehicle information from the violation vehicle, the real-time automatic detection of the vehicle violation and the extraction of the violation vehicle information are realized, the whole detection process is free from manual intervention, the detection efficiency is high, each frame is identified, missing frames are avoided, and missing detection in a snapshot gap is avoided.

Further, when the extraction of the information of the illegal vehicles fails, the corresponding vehicle information is acquired from the cached video image through the identity of the illegal vehicles, so that the accuracy and the effectiveness of the extraction of the information of the illegal vehicles are ensured.

Further, the vehicle and lane line identification is carried out on the ith frame image and the i-1 th frame image in sequence, and the identification of each vehicle is generated for each identified vehicle according to the vehicle attribute, so that the identification of each vehicle in the video image is unique.

Further, the vehicle segmentation matching image and the lane line segmentation matching image are obtained by carrying out similarity comparison on each vehicle in the vehicle segmentation image and each lane line in the lane line segmentation image of the i-1 frame image and each vehicle in the vehicle segmentation image and each lane line in the lane line segmentation image of the i-1 frame image, so that the algorithm is simple, the operation efficiency is high, and the vehicles and the lane lines can be matched quickly and accurately.

Further, whether the vehicle breaks rules or not is judged according to the lane attribute of each lane in the lane segmentation matching image and by combining the vehicle matching line in the vehicle segmentation matching image, automatic detection of the vehicle breaks rules or rules can be achieved through one frame of image or two continuous frames of images, detection efficiency is high, and detection results are accurate.

Further, when the video images exceed the storage capacity, deleting at least part of the video images stored first, deleting the video images stored first, not affecting the real-time violation detection result, and releasing the storage space.

Further, the vehicle information related to the violation reporting and the ith frame image or the ith frame image and the ith-1 frame image are stored, so that the violation evidence can be effectively stored.

Further, the stored vehicle information related to the violation reporting and the stored ith frame image or the ith frame image and the ith-1 frame image are uploaded to a remote server every preset time or when preset conditions are met, the local storage is deleted after the uploading is completed, the uploading of the violation evidence can be completed when the system is idle, the success rate of uploading is guaranteed, the local storage is deleted after the uploading is successful, and the local storage space is further released.

Drawings

FIG. 1 is a flow chart of a real-time vehicle violation detection method according to an embodiment of the present invention;

FIG. 2 is a flow chart of a vehicle matching process in one embodiment of the invention;

FIG. 3 is a schematic diagram of a real-time vehicle violation detection device according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of the structure of a real-time vehicle violation detection device of an embodiment of the invention;

fig. 5 is a schematic structural view of a storage medium according to an embodiment of the present invention.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

The terms "first," "second," "third," and the like in this application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", and "a third" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise. All directional indications (such as up, down, left, right, front, back … …) in the embodiments of the present application are merely used to explain the relative positional relationship, movement, etc. between the components in a particular gesture (as shown in the drawings), and if the particular gesture changes, the directional indication changes accordingly. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.

FIG. 1 is a flow chart of a real-time vehicle violation detection method according to an embodiment of the invention. The real-time vehicle violation detection method can be operated on a vehicle-mounted intelligent device, the vehicle-mounted intelligent device can be installed on a vehicle (such as a bus, a traffic assistant vehicle, a private car and the like), video images of a lane and a vehicle at the front and the rear can be obtained, and the vehicle violation can be detected in real time according to the video images. The method and the device can be applied to intelligent traffic scenes, so that construction of intelligent cities is promoted. It should be noted that, if there are substantially the same results, the method of the present invention is not limited to the flow sequence shown in fig. 1. As shown in fig. 1, the method comprises the steps of:

Step S11: and acquiring a real-time video image.

In this embodiment, the video image is obtained by shooting the vehicle-mounted intelligent device in the driving process, and the vehicle-mounted intelligent device may be a shooting device with a driving shooting recording function, such as a vehicle-mounted driving recorder.

Step S12: and caching the video image.

In step S12, the video images are stored in real time, and when the video images exceed the storage capacity, at least part of the video images stored first can be deleted; in this embodiment, the at least part of the video images may be one frame of video images, that is, when the storage capacity is full, each time a frame of video image at the current moment is newly stored, the earliest frame of video image stored in the storage capacity is deleted.

Step S13: and carrying out vehicle and lane line identification on an ith frame image and an ith-1 frame image in the video image, generating identification marks for each identified vehicle, and carrying out vehicle and lane segmentation and matching on the ith frame image and the ith-1 frame image according to identification results to obtain a vehicle segmentation matching image and a lane segmentation matching image, wherein i is a natural number greater than or equal to 2.

Specifically, step S13 includes the steps of:

step S131: each lane and its lane type, each vehicle and its vehicle attribute in each frame image is identified, the vehicle attribute includes one, two or more of color, vehicle model, brand, license plate, the lane type includes one, two or more of real line lane, diversion lane, bus lane.

It may be appreciated that in step S131, the image recognition technology may be used to identify the vehicle and the lane line as the target objects, and each lane and its lane type, each vehicle and its vehicle attribute in each frame of image may be obtained through the identification, where in this embodiment, the vehicle attribute includes one, two or more of a color, a vehicle model, a brand, and a license plate, and the lane type includes one, two or more of a real lane, a diversion lane, and a bus lane.

Step S132: and generating the identity mark of each vehicle according to the vehicle attribute.

According to one embodiment of the present invention, the identity of each vehicle may be generated according to the vehicle attribute, for example, an attribute value and a weight are set for each attribute in the vehicle attribute, and the identity is calculated through the attribute value and the weight, for example: the vehicle attribute of a vehicle is red and a car, the attribute value of "red" is set to be 10, the weight is 0.4, the attribute value of "car" is set to be 20, and the weight is 0.6, so that the identity mark 10 x 0.4+20 x 0.6=16 can be generated for the vehicle according to the attribute value, and the above is an illustration of an embodiment, and other similar calculation methods can be adopted for different vehicle attributes, and are not illustrated one by one.

Step S133: and dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image.

Through an image recognition technology, the vehicle and the lane line can be respectively recognized in each frame of image, the vehicle and the lane line are segmented from the image, and the vehicle segmentation image and the lane segmentation image are respectively obtained.

Step S134: and carrying out vehicle matching on the vehicle segmentation image of the ith frame image and the i-1 th frame image to obtain the vehicle segmentation matching image.

Specifically, each vehicle in the vehicle segmentation image of the i-1 frame image and each vehicle in the vehicle segmentation image of the i frame image are subjected to similarity comparison, and a first similarity comparison result is obtained, wherein the first similarity comparison result comprises that each vehicle of the i-1 frame image and at least one vehicle of the i frame image are similar vehicles; in this embodiment, the similarity comparison between each vehicle in the vehicle segmentation image of the i-1 th frame image and each vehicle in the vehicle segmentation image of the i-1 th frame image may be performed according to each vehicle in each frame image identified in step S131 and the vehicle attribute thereof, and a first similarity comparison result may be obtained according to the similarity of the vehicle attributes.

Randomly matching each vehicle of the i-1 frame image with one of the similar vehicles corresponding to the i frame image or carrying out residual matching with the unmatched vehicle in the i frame image; in this embodiment, each vehicle of the i-1 frame image is randomly matched with one of the similar vehicles corresponding to the i frame image, and the random matching result needs to satisfy the one-to-one correspondence between the vehicle of the i-1 frame image and the similar vehicle corresponding to the i frame image, that is, the multiple vehicles of the i-1 frame image cannot simultaneously correspond to the same similar vehicle in the i frame image, the same vehicle of the i-1 frame image cannot simultaneously correspond to the multiple similar vehicles in the i frame image, or the remaining vehicles of the i-1 frame image and the unmatched vehicles in the i frame image are subjected to the remaining matching.

Further, whether the remaining matching result is reasonable is determined according to the first similarity comparison result, in this embodiment, whether the remaining matching result is reasonable is determined, that is, whether the remaining matching result of the i-1 th frame image and the remaining matching result of the unmatched vehicle in the i-th frame image satisfy the first similarity comparison result is determined, that is, the vehicle attribute of the remaining matched vehicle also satisfies the similarity relationship in the first similarity comparison result. If the comparison result is reasonable, that is, the first similarity comparison result is met, the i-th frame image and the vehicle segmentation images of the i-1 th frame image are spliced together, matched vehicles are loaded on a matching line, and therefore the vehicle segmentation matching image is obtained, if the comparison result is not reasonable, the vehicle of the i-1 th frame image and other similar vehicles corresponding to the i-th frame image are matched until the rest matching result is judged to be reasonable, the vehicle segmentation images of the i-th frame image and the i-1 th frame image are spliced together, and the matched vehicles are loaded on the matching line, so that the vehicle segmentation matching image is obtained.

The above-described vehicle matching process is described below with an example. Referring to fig. 2, fig. 2 is a schematic flow chart of a vehicle matching process in an embodiment of the present invention, in which each vehicle in the vehicle segmentation image of the i-1 st frame image is compared with each vehicle in the vehicle segmentation image of the i-1 st frame image, in this embodiment, a first similarity comparison result is obtained by comparing the vehicle attribute, where the first similarity comparison result includes that an x1 vehicle of the i-1 st frame image is similar to a y1 vehicle and a y2 vehicle of the i-1 st frame image, an x2 vehicle of the i-1 st frame image is similar to a y2 vehicle and a y3 vehicle of the i-1 st frame image, and an x3 vehicle of the i-1 st frame image is similar to a y1 vehicle of the current image, as shown in fig. 2 (a).

Further, carrying out random matching on the x1, x2 and x3 vehicles of the i-1 frame image and the similar vehicles corresponding to the i-1 frame image or carrying out residual matching on the residual vehicles of the i-1 frame image and the unmatched vehicles in the i-1 frame image, wherein a random matching result needs to meet the requirement that the vehicles of the i-1 frame image and the similar vehicles corresponding to the i-1 frame image are in one-to-one correspondence, if the x1 vehicles randomly match the unmatched y1 vehicles in the similar vehicles corresponding to the x1 vehicles, the x2 vehicles can only match the unmatched residual vehicles y3 vehicles in the similar vehicles corresponding to the x2 vehicles, the unmatched residual vehicles y3 vehicles are not in the first similarity comparison result according to the first similarity comparison result, then a re-match is required, in this example, the vehicle of the i-1 th frame image is matched with one of the similar vehicles corresponding to the i-1 th frame image until the remaining matching result is judged to be reasonable, as shown in fig. 2 (b), the x1 vehicle is matched with another vehicle of y1 and y2, such as the x1 vehicle is matched with the y2 vehicle, the x2 is matched with another vehicle which is not matched and is similar to x2, such as the x2 vehicle is matched with the y3 vehicle, the x3 is matched with the y1 vehicle which is not matched and is similar to x3, namely, the similar match of each vehicle in the i-1 th frame image and the vehicle of the i-1 th frame image is completed, the vehicle segmentation images of the i-1 th frame image and the i-1 th frame image are spliced together and the matched vehicles are loaded with matching lines, thereby obtaining the vehicle division matching image.

In the matching process, if the number of each vehicle in the vehicle segmentation image of the i-1 frame image is different from the number of each vehicle in the vehicle segmentation image of the i-1 frame image, if the number of each vehicle in the vehicle segmentation image of the i-1 frame image is x1, x2, x3, and the number of each vehicle in the vehicle segmentation image of the i-1 frame image is y1, y2, it can be considered that one vehicle in the vehicle segmentation image of the i-1 frame image leaves the shooting picture, and in the matching process, only the similarity comparison and matching of the vehicles y1, y2 in the vehicle segmentation image of the i-1 frame image are required to be ensured; if the vehicles in the vehicle segmentation image of the i-1 frame image are x1 and x2, and the vehicles in the vehicle segmentation image of the i frame image are y1, y2 and y3, then it can be considered that one vehicle of the i frame image enters the shooting picture, and in the matching process, only the similarity comparison and matching of the vehicles x1 and x2 in the vehicle segmentation image of the i-1 frame image are required to be ensured, and the unmatched vehicles in the vehicle segmentation image of the i frame image can be subjected to the unmatched and matched in the vehicles in the vehicle segmentation image of the next frame, namely the i+1st frame image.

Step S135: and carrying out lane matching on the lane segmentation image of the ith frame image and the i-1 th frame image to obtain the lane segmentation matching image.

The lane segmentation matching image obtaining process is similar to the vehicle segmentation matching image obtaining process in the step S134, namely, each lane in the lane segmentation image of the i-1 th frame image is compared with each lane in the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that each lane of the i-1 th frame image and at least one lane of the i-1 th frame image are similar lanes; in this embodiment, the similarity comparison between each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-1 th frame image may be performed according to each lane and the lane type thereof in each frame image identified in step S131, and a second similarity comparison result may be obtained according to the lane type.

Randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the i-1 th frame image or carrying out residual matching with the unmatched lanes in the i-1 th frame image; in this embodiment, each lane of the i-1 frame image is randomly matched with one of the similar lanes corresponding to the i frame image, and the random matching result needs to satisfy the one-to-one correspondence between the lanes of the i-1 frame image and the similar lanes corresponding to the i frame image, that is, multiple lanes of the i-1 frame image cannot simultaneously correspond to the same similar lane in the i frame image, the same lane of the i-1 frame image cannot simultaneously correspond to multiple similar lanes in the i frame image, or residual matching is performed on residual lanes of the i-1 frame image and unmatched lanes in the i frame image.

Further, whether the remaining matching result is reasonable is determined according to the second similarity comparison result, in this embodiment, whether the remaining matching result is reasonable is determined, that is, whether the remaining matching result of the i-1 th frame image and the remaining matching result of the non-matching lane in the i-th frame image satisfy the second similarity comparison result is determined, that is, the lane type of the remaining matching lane also satisfies the similarity relationship in the second similarity comparison result. If the comparison result is reasonable, namely the second similarity comparison result is met, the i-th frame image and the lane segmentation images of the i-1 th frame image are spliced together, matched lanes are loaded on a matching line, so that the lane segmentation matching image is obtained, if the comparison result is not reasonable, the lanes of the i-1 th frame image and other similar lanes corresponding to the i-th frame image are matched until the rest matching result is judged to be reasonable, the lane segmentation images of the i-th frame image and the i-1 th frame image are spliced together, and the matched lanes are loaded on the matching line, so that the lane segmentation matching image is obtained.

The lane matching process described above is also described below with an example. And carrying out similarity comparison on each lane in the lane segmentation image of the i-1 frame image and each lane in the lane segmentation image of the i-1 frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that an m1 lane of the i-1 frame image is similar to n1 lanes and n2 lanes of the i-1 frame image, an m2 lane of the i-1 frame image is similar to n2 lanes and n3 lanes of the i-1 frame image, and an m3 lane of the i-1 frame image is similar to n1 lanes of the current image.

And randomly matching m1, m2 and m3 lanes of the i-1 frame image with the corresponding similar lanes of the i-1 frame image or performing residual matching on the residual lanes of the i-1 frame image with the unmatched lanes in the i-1 frame image, if the m1 lanes are matched with the n1 lanes, the m2 lanes are matched with the n2 lanes, the m3 is matched with the residual n3 lanes, and according to the second similarity comparison result, the m3 lanes are matched with the n3 lanes and need to be matched again, in the case of re-matching, the m1 lanes are matched with the n2 lanes, the m2 lanes are matched with the n3 lanes, and the m3 lanes are matched with the n1 lanes, namely, the similar matching of each lane in the i-1 frame image and the lanes of the i-1 frame image is completed, the i-1 frame image and the segmented images of the i-1 frame image are spliced together, and the matched lanes are loaded, so that the matched lines of the lanes are obtained.

Similarly, in the matching process, if the number of lanes in the lane segmentation image of the i-1 th frame image is different from the number of lanes in the lane segmentation image of the i-1 th frame image, if the number of lanes in the lane segmentation image of the i-1 th frame image is m1, m2, and m3, the number of lanes in the lane segmentation image of the i-1 th frame image is n1 and n2, it can be considered that one journey in the lanes in the lane segmentation image of the i-1 th frame image is terminated, and in the matching process, only the similarity comparison and matching of the lanes n1 and n2 in the lane segmentation image of the i-th frame image are required to be ensured; if the lanes in the lane segmentation image of the i-1 frame image are m1 and m2, and the lanes in the lane segmentation image of the i frame image are n1, n2 and n3, then it can be considered that a new lane exists in the lanes in the lane segmentation image of the i frame image, and in the matching process, only the similarity comparison and matching of the lanes m1 and m2 in the lane segmentation image of the i-1 frame image are ensured, and the unmatched lanes in the lane segmentation image of the i frame image can be compared and matched in the lanes in the lane segmentation image of the next frame, i.e. the i+1st frame image.

Step S14: judging whether a vehicle violation exists in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; if it is determined that there is a vehicle violation, step S15 is performed.

It can be understood that if no vehicle violation is determined in the i frame image according to the vehicle segmentation matching image and the lane segmentation matching image, the same method can be continuously adopted to perform violation determination on the subsequent i+1st frame image.

In step S14, whether there is a vehicle violation in the ith frame image may be determined according to the lane attribute of each lane in the lane segmentation matching image and by combining with the vehicle matching line in the vehicle segmentation matching image, if there is a lane with a preset lane attribute in the lane segmentation matching image and the vehicle matching line extends to the lane with the preset lane attribute and other lanes, it is determined that the vehicle corresponding to the vehicle matching line is a violation vehicle.

In this embodiment, the preset lane attribute includes one, two or more of a real lane, a diversion lane and a bus lane, when the lane attribute of the lane in the lane segmentation matching image is identified to have the preset lane attribute, whether the vehicle violates rules or not is determined by detecting whether the vehicle matching line of the vehicle extends to the lane with the preset lane attribute and other lanes, three lanes including the real lane, the diversion lane and the bus lane are respectively described below, and when the lane attribute of the lane in the lane segmentation matching image is identified to be the real lane, if the vehicle matching line of the vehicle extends from one side lane to the other side lane of the real lane, the vehicle can be regarded as crossing the real lane and causing rules violations; when the lane attribute of the lane in the lane segmentation matching image is identified as a diversion lane, if the vehicle matching line of a vehicle extends from the lane outside the diversion lane to the inside of the diversion lane, the vehicle can be considered to occupy the diversion lane, and thus the violation is caused; when the lane attribute of the lane in the lane segmentation matching image is identified as a bus lane, if the vehicle matching line of a vehicle extends from the left lane to the right lane of the bus lane, the vehicle can be considered to occupy the bus lane, and thus the violation is caused.

Step S15: and extracting vehicle information of the illegal vehicle according to the ith frame image.

When the vehicle violation is detected, vehicle information of the vehicle with the violation can be extracted according to the ith frame of image. In this embodiment, the vehicle information of the violation vehicle may be a license plate number of the violation vehicle.

Step S16: and judging whether the vehicle information extraction of the illegal vehicle is successful, if so, executing the step S17, and if not, executing the step S18.

It should be noted that, due to some reasons, such as too far distance or obstacle shielding, when the vehicle breaks rules, the vehicle information of the vehicle breaking rules in the i-th frame image may be blurred or hidden, and the vehicle information of the vehicle breaking rules cannot be extracted from the i-th frame image. Therefore, it is necessary to determine whether the extraction of the vehicle information of the offending vehicle is successful.

Step S17: and transmitting the extracted vehicle information of the violation vehicle, the ith frame image or the vehicle information, the ith frame image and the ith-1 frame image, and reporting the violation.

If the vehicle information of the violation vehicle is successfully extracted, the vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image are stored; and uploading the stored vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image to a remote server every preset time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image after the uploading is completed. In this embodiment, the predetermined time may be set to be a time when the system is idle, such as uploading the cached violation record at night; the preset condition can also be set as uploading the cached violation record when no vehicle violation is detected in a period of time.

Step S18: acquiring the identity of the illegal vehicle; and acquiring corresponding vehicle information from the cached video images according to the identity, and sending the corresponding vehicle information and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image for illegal reporting.

If the extraction of the vehicle information of the violation vehicle fails, the corresponding vehicle information needs to be acquired from the cached video image through the identity of the violation vehicle, and in this embodiment, the vehicle information may be a license plate number. Similarly, after the vehicle information is acquired, the vehicle information and the ith frame image related to the violation report or the vehicle information and the ith frame image and the ith-1 frame image can be stored; and uploading the stored vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image to a remote server every preset time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image after the uploading is completed.

According to the real-time vehicle violation detection method, the video image is obtained in real time, the ith frame image and the ith-1 frame image are sequentially subjected to vehicle and lane line identification, identification is generated for each identified vehicle, segmentation and matching of the vehicle and the lane are carried out on the ith frame image and the ith-1 frame image according to the identification result, the vehicle segmentation matching image and the lane segmentation matching image are obtained, whether the vehicle is in the ith frame image or not is judged according to the vehicle segmentation matching image and the lane segmentation matching image, and the vehicle information of the violation is extracted, so that real-time automatic detection of the vehicle violation and the vehicle information of the violation are realized, the whole detection process is free from manual intervention, each frame is identified, missing frames are avoided, and missing detection in a snapshot gap is avoided.

Fig. 3 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the present invention. As shown in fig. 3, the real-time vehicle violation detection device 20 includes a video receiving module 21, a video buffering module 22, an information extracting module 23, a violation judging module 24, a violation information extracting module 25, a violation information storing module 26 and a violation information transmitting module 27.

A video receiving module 21 for acquiring real-time video images; in this embodiment, the video receiving module 21 may receive a video image captured by the vehicle-mounted camera during the driving process.

A video caching module 22, configured to cache the video image acquired by the video receiving module 21; at least a portion of the video images that were first stored may be deleted when the video images exceed the storage capacity of the video cache module 22.

The information extraction module 23 is configured to identify a vehicle and a lane line in the i-th frame image and the i-1-th frame image in the video image acquired by the video receiving module 21, generate an identity for each identified vehicle, and segment and match the i-th frame image and the i-1-th frame image according to the identification result to obtain a vehicle segmentation matching image and a lane segmentation matching image, where i is a natural number greater than or equal to 2, where the information extraction module 23 may respectively identify the vehicle and the lane line as target objects by using an image identification technology, and acquire each lane and a lane type thereof, each vehicle and a vehicle attribute thereof in each frame image by identifying, where in this embodiment, the vehicle attribute includes one, two or more of a color, a vehicle model, a brand, and a license plate, and the lane type includes one, two or more of a real lane, a diversion lane, and a bus lane. And generating the identity of each vehicle according to the vehicle attribute.

The information extraction module 23 may identify the vehicle and the lane line in each frame of image, and segment the vehicle and the lane line from the image, thereby obtaining the vehicle segmentation image and the lane segmentation image.

Specifically, each vehicle in the vehicle segmentation image of the i-1 frame image and each vehicle in the vehicle segmentation image of the i frame image are subjected to similarity comparison, and a first similarity comparison result is obtained, wherein the first similarity comparison result comprises that each vehicle of the i-1 frame image and at least one vehicle of the i frame image are similar vehicles; randomly matching each vehicle of the i-1 frame image with one of the similar vehicles corresponding to the i frame image or carrying out residual matching with the unmatched vehicle in the i frame image;

judging whether the residual matching result is reasonable or not according to the first similarity comparison result, if so, splicing the ith frame image and the vehicle segmentation images of the ith-1 frame image together, loading the matched vehicles on a matching line, so as to obtain the vehicle segmentation matching image, if not, matching the vehicle of the ith-1 frame image with other similar vehicles corresponding to the ith frame image until the residual matching result is judged to be reasonable, and splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together, and loading the matched vehicles on the matching line, so as to obtain the vehicle segmentation matching image.

The information extraction module 23 performs similarity comparison on each lane in the lane segmentation image of the i-1 th frame image and each lane in the lane segmentation image of the i-1 th frame image to obtain a second similarity comparison result, wherein the second similarity comparison result comprises that each lane of the i-1 th frame image and at least one lane of the i-1 th frame image are similar lanes; randomly matching each lane of the i-1 th frame image with one of the similar lanes corresponding to the i-1 th frame image or carrying out residual matching with the unmatched lanes in the i-1 th frame image,

judging whether the residual matching result is reasonable or not according to the second similarity comparison result; if the matching is reasonable, the lane segmentation images of the ith frame image and the ith-1 frame image are spliced together, and matched lanes are loaded on a matching line, so that the lane segmentation matching image is obtained, if the matching is not reasonable, the lanes of the ith-1 frame image and other similar lanes corresponding to the ith frame image are matched until the residual matching result is judged to be reasonable, and if the lane segmentation images of the ith frame image and the ith-1 frame image are spliced together, and matched lanes are loaded on the matching line, so that the lane segmentation matching image is obtained.

A violation judging module 24, configured to judge whether there is a vehicle violation in the i-th frame image according to the vehicle segmentation matching image and the lane segmentation matching image extracted by the information extracting module 23; when it is determined that there is a vehicle violation, vehicle information of the vehicle violation is extracted according to the ith frame image, specifically, the violation determination module 24 determines whether there is a vehicle violation in the ith frame image, and may determine whether there is a vehicle violation according to the lane attribute of each lane in the lane segmentation matching image and in combination with the vehicle matching line in the vehicle segmentation matching image, if the lane in the lane segmentation matching image has a preset lane attribute, and if there is a vehicle matching line extending to the lane with the preset lane attribute and other lanes, it determines that the vehicle corresponding to the vehicle matching line is a vehicle violation. When the vehicle violation is detected, vehicle information of the vehicle with the violation can be extracted according to the ith frame of image. In this embodiment, the vehicle information of the violation vehicle may be a license plate number of the vehicle.

If the rule-breaking judgment module 24 successfully extracts the vehicle information of the rule-breaking vehicle, the vehicle information and the ith frame image related to the rule-breaking report or the vehicle information and the ith frame image and the ith-1 frame image are sent to the rule-breaking information storage module 26 for storage; the violation information sending module 27 stores the stored vehicle information and the i-th frame image or the vehicle information and the i-th frame image and the i-th-1-th frame image related to the violation report from the violation information storage module 26 to a remote server every predetermined time or when a preset condition is met, and deletes the vehicle information and the i-th frame image or the vehicle information and the i-th frame image and the i-1-th frame image related to the violation report stored in the violation information storage module 26 after the uploading is completed.

If the extraction of the information of the offending vehicle by the offending judging module 24 fails, the identity of the offending vehicle is sent to the offending information extracting module 25, and the offending information extracting module 25 obtains corresponding vehicle information in the video image cached by the video receiving module 21 according to the identity of the offending vehicle, where in this embodiment, the vehicle information may be a license plate number. Likewise, after the vehicle information is acquired by the violation information extraction module 25, the vehicle information and the i frame image related to the violation report or the vehicle information and the i frame image and the i-1 frame image may be sent to the violation information storage module 26 for storage; the violation information sending module 27 stores the stored vehicle information and the i-th frame image or the vehicle information and the i-th frame image and the i-th-1-th frame image related to the violation report from the violation information storage module 26 to a remote server every predetermined time or when a preset condition is met, and deletes the vehicle information and the i-th frame image or the vehicle information and the i-th frame image and the i-1-th frame image related to the violation report stored in the violation information storage module 26 after the uploading is completed.

It can be understood that the specific manner of implementing each function by each module of the real-time vehicle violation detection device can refer to the specific steps corresponding to the above embodiment, so that the description thereof will not be repeated here.

Referring to fig. 4, fig. 4 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the present invention. As shown in fig. 4, the real-time vehicle violation detection device 30 comprises a memory 32, a processor 31 and a computer program stored on the memory 32 and executable on the processor 31, the processor 31 implementing the following steps when executing the computer program: acquiring a real-time video image; caching the video image; identifying vehicle and lane lines of an ith frame image and an ith-1 frame image in the video image, generating identification marks for each identified vehicle, and dividing and matching the ith frame image and the ith-1 frame image according to identification results to obtain a vehicle division matching image and a lane division matching image, wherein i is a natural number greater than or equal to 2; judging whether a vehicle violation exists in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; if the vehicle violations are judged, extracting the information of the violating vehicles according to the ith frame image; if the information of the illegal vehicles is successfully extracted, the extracted information of the illegal vehicles and the ith frame image or the ith frame image and the ith-1 frame image are sent to report the illegal vehicles; if the information extraction of the illegal vehicles fails, the identity of the illegal vehicles is obtained; and acquiring corresponding vehicle information from the cached video images according to the identity, and transmitting the corresponding vehicle information and the ith frame image or the corresponding vehicle information and the ith frame image and the ith-1 frame image for illegal reporting.

In one embodiment, the steps of identifying the vehicle and the lane line of the ith frame image and the ith-1 frame image in the video image, generating an identity for each identified vehicle, and dividing and matching the vehicle and the lane of the ith frame image and the ith-1 frame image according to the identification result, and obtaining the vehicle division matching image and the lane division matching image comprise: identifying each lane and lane type thereof, each vehicle and vehicle attribute thereof in each frame of image, wherein the vehicle attribute comprises one, two or more of color, vehicle type, brand and license plate, and the lane type comprises one, two or more of real line road, diversion line road and bus lane; generating an identity of each vehicle according to the vehicle attribute; dividing each frame of image according to the identification result to obtain a vehicle division image and a lane division image; performing vehicle matching on the vehicle segmentation image of the ith frame image and the i-1 th frame image to obtain a vehicle segmentation matching image; and carrying out lane matching on the lane segmentation image of the ith frame image and the i-1 th frame image to obtain the lane segmentation matching image.

In one embodiment, the step of obtaining the vehicle segmentation matching image by performing vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1 th frame image includes: performing similarity comparison on each vehicle in the vehicle segmentation image of the i-1 frame image and each vehicle in the vehicle segmentation image of the i frame image to obtain a first similarity comparison result, wherein each vehicle in the i-1 frame image and at least one vehicle in the i frame image are similar vehicles; randomly matching each vehicle of the i-1 frame image with one of the similar vehicles corresponding to the i frame image or carrying out residual matching with the unmatched vehicle in the i frame image; judging whether the residual matching result is reasonable or not according to the first similarity comparison result, if so, splicing the ith frame image and the vehicle segmentation images of the ith-1 frame image together, loading the matched vehicles on a matching line, so as to obtain the vehicle segmentation matching image, if not, matching the vehicle of the ith-1 frame image with other similar vehicles corresponding to the ith frame image until the residual matching result is judged to be reasonable, and splicing the vehicle segmentation images of the ith frame image and the ith-1 frame image together, and loading the matched vehicles on the matching line, so as to obtain the vehicle segmentation matching image.

In one embodiment, the step of performing lane matching on the lane segmentation images of the i-th frame image and the i-1 th frame image to obtain the lane segmentation matching image includes: carrying out similarity comparison on each lane in the lane segmentation image of the i-1 frame image and each lane in the lane segmentation image of the i frame image to obtain a second similarity comparison result, wherein each lane of the i-1 frame image and at least one lane of the i frame image are similar lanes; randomly matching each lane of the i-1 frame image with one of the similar lanes corresponding to the i frame image or carrying out residual matching with the unmatched lane in the i frame image, and judging whether the residual matching result is reasonable or not according to the second similarity comparison result; if the matching is reasonable, the lane segmentation images of the ith frame image and the ith-1 frame image are spliced together, and matched lanes are loaded on a matching line, so that the lane segmentation matching image is obtained, if the matching is not reasonable, the lanes of the ith-1 frame image and other similar lanes corresponding to the ith frame image are matched until the residual matching result is judged to be reasonable, and if the lane segmentation images of the ith frame image and the ith-1 frame image are spliced together, and matched lanes are loaded on the matching line, so that the lane segmentation matching image is obtained.

In one embodiment, the step of determining whether there is a vehicle violation in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image includes: judging whether a vehicle violates regulations or not according to the lane attribute of each lane in the lane segmentation matching image and combining the vehicle matching line in the vehicle segmentation matching image, and judging that the vehicle corresponding to the vehicle matching line is a violation vehicle if the lane with the preset lane attribute in the lane segmentation matching image and the vehicle matching line extends to the lane with the preset lane attribute and other lanes.

In one embodiment, the step of buffering the video image includes: deleting at least part of the video images stored first when the video images exceed a storage capacity; the at least part of the video image is a frame of video image.

In one embodiment, the processor 31, when executing the computer readable instructions, further performs the steps of: storing the vehicle information related to the violation report, the ith frame image or the corresponding vehicle information, the ith frame image and the ith-1 frame image; and storing and uploading the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the ith-1 frame image to a remote server every preset time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the ith-1 frame image after the uploading is completed.

Referring to fig. 5, fig. 5 is a schematic diagram illustrating a structure of a storage medium according to an embodiment of the present invention. A storage medium storing computer-readable instructions 41 as shown in fig. 5, which computer-readable instructions 41, when executed by one or more processors, cause the one or more processors to perform the steps of: acquiring a real-time video image; caching the video image; identifying vehicle and lane lines of an ith frame image and an ith-1 frame image in the video image, generating identification marks for each identified vehicle, and dividing and matching the ith frame image and the ith-1 frame image according to identification results to obtain a vehicle division matching image and a lane division matching image, wherein i is a natural number greater than or equal to 2; judging whether a vehicle violation exists in the ith frame of image according to the vehicle segmentation matching image and the lane segmentation matching image; if the vehicle violations are judged, extracting the information of the violating vehicles according to the ith frame image; if the information of the illegal vehicles is successfully extracted, the extracted information of the illegal vehicles and the ith frame image or the ith frame image and the ith-1 frame image are sent to report the illegal vehicles; if the information extraction of the illegal vehicles fails, the identity of the illegal vehicles is obtained; and acquiring corresponding vehicle information from the cached video images according to the identity, and transmitting the corresponding vehicle information and the ith frame image or the corresponding vehicle information and the ith frame image and the ith-1 frame image for illegal reporting.

In one embodiment, the processor when executing the computer readable instructions 41 further performs the steps of: storing the vehicle information related to the violation report, the ith frame image or the corresponding vehicle information, the ith frame image and the ith-1 frame image; and storing and uploading the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the ith-1 frame image to a remote server every preset time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the ith frame image or the corresponding vehicle information and the ith frame image and the ith-1 frame image after the uploading is completed.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored in a computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).

The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims

1. The real-time vehicle violation detection method is characterized by comprising the following steps of:

acquiring a real-time video image;

caching the video image;

2. The method for detecting real-time vehicle violations according to claim 1, wherein the steps of identifying the vehicle and the lane lines of the ith frame image and the ith-1 frame image in the video image, generating the identity of each identified vehicle, and dividing and matching the vehicle and the lane of the ith frame image and the ith-1 frame image according to the identification result, and obtaining the vehicle division matching image and the lane division matching image comprise:

generating an identity of each vehicle according to the vehicle attribute;

3. The real-time vehicle violation detection method of claim 2, wherein performing vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1 th frame image to obtain the vehicle segmentation matching image, comprises:

randomly matching each vehicle of the i-1 frame image with one of the similar vehicles corresponding to the i frame image or carrying out residual matching with the unmatched vehicle in the i frame image;

judging whether the residual matching result is reasonable or not according to the first similarity comparison result,

4. The real-time vehicle violation detection method of claim 2, wherein performing lane matching on the i-th frame image and the lane-divided image of the i-1 th frame image to obtain the lane-divided matching image includes:

judging whether the residual matching result is reasonable or not according to the second similarity comparison result;

5. The method according to claim 2, wherein the determining whether there is a vehicle violation in the i-th frame image according to the vehicle segmentation matching image and the lane segmentation matching image includes:

6. The method of real-time vehicle violation detection of claim 1, wherein the caching the video image further comprises:

7. The real-time vehicle violation detection method of claim 1, further comprising:

storing the vehicle information related to the violation report or the ith frame image or the vehicle information and the ith frame image and the ith-1 frame image; a kind of electronic device with high-pressure air-conditioning system

And uploading the stored vehicle information related to the violation report and the i-th frame image or the vehicle information and the i-th frame image and the i-1-th frame image to a remote server every preset time or when a preset condition is met, and deleting the stored vehicle information related to the violation report and the i-th frame image or the vehicle information and the i-th frame image and the i-1-th frame image after the uploading is completed.

8. A real-time vehicle violation detection device, comprising:

the video receiving module is used for acquiring real-time video images;

the video caching module is used for caching the video image;

9. A real-time vehicle violation detection device, characterized in that the real-time vehicle violation detection device comprises a processor, a memory coupled to the processor, wherein,

the memory has stored therein computer readable instructions which, when executed by the processor, cause the processor to perform the steps of the real-time vehicle violation detection method of any of claims 1 to 7.

10. A storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the real-time vehicle violation detection method of any of claims 1-7.