WO2021120776A1 - 实时车辆违章检测方法、装置、设备及存储介质 - Google Patents

实时车辆违章检测方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021120776A1
WO2021120776A1 PCT/CN2020/118438 CN2020118438W WO2021120776A1 WO 2021120776 A1 WO2021120776 A1 WO 2021120776A1 CN 2020118438 W CN2020118438 W CN 2020118438W WO 2021120776 A1 WO2021120776 A1 WO 2021120776A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
image
frame image
lane
matching
Prior art date
Application number
PCT/CN2020/118438
Other languages
English (en)
French (fr)
Inventor
芦文峰
刘伟超
郭倜颖
曾凡涛
陈远旭
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021120776A1 publication Critical patent/WO2021120776A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
    • G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • This application relates to the field of video processing technology, and in particular to a real-time vehicle violation detection method, device, equipment and storage medium.
  • Intelligent traffic violation detection is to use various sensors and image acquisition equipment, combined with the back-end violation judgment algorithm and information extraction algorithm, to automatically judge the violation of the vehicle.
  • This type of technology can help traffic management departments improve the speed and accuracy of judging violations, while reducing labor costs, and avoiding errors such as false detections and missed detections caused by human causes.
  • the other is to obtain driving video through a vehicle-mounted camera, and then process one or a few frames in the video locally, or send it to a remote server for processing for violation judgment.
  • processing one or a few frames in the video locally may cause difficulty in extracting vehicle information, such as the license plate in the extracted image is blocked, etc.; sending to a remote server for processing increases the download burden of the server and cannot be large-scale use.
  • This application provides a real-time vehicle violation detection method, device, equipment, and storage device, which can achieve real-time automatic violation detection and accurately extract vehicle violation information.
  • a technical solution adopted in this application is to provide a real-time vehicle violation detection method, which includes the following steps:
  • I-1 frame image segmentation and matching of vehicles and lanes to obtain vehicle segmentation matching images and lane segmentation matching images, where i is a natural number greater than or equal to 2;
  • a technical solution adopted by this application is to provide a real-time vehicle violation detection device, which includes:
  • the video receiving module is used to obtain real-time video images
  • the video buffer module is used to buffer the video image
  • the information extraction module is used for recognizing vehicles and lane lines on the i-th frame image and the i-1th frame image in the video image, generating an identity mark for each recognized vehicle, and identifying the i-th frame image according to the recognition result. Segment and match the frame image and the i-1th frame image of vehicles and lanes to obtain vehicle segmentation matching images and lane segmentation matching images, where i is a natural number greater than or equal to 2;
  • Violation judgment module for judging whether there is a vehicle violation in the i-th frame image according to the vehicle segmentation matching image and the lane segmentation matching image; when it is judged that there is a vehicle violation, the vehicle is in violation of the i-th frame image Vehicle information extraction;
  • the violation information extraction module is used to obtain the identity of the violating vehicle when the violation judgment module fails to extract the vehicle information of the violating vehicle; and obtain the corresponding information from the cached video image according to the identity.
  • Vehicle information ;
  • a violation information storage module for storing the vehicle information involved in the violation report and the i-th frame image or the vehicle information, the i-th frame image and the i-1th frame image;
  • the violation information sending module is used to send the vehicle information involved in the violation report and the i-th frame image or the vehicle information, the i-th frame image and the i-1th frame image for violation report.
  • a technical solution adopted by this application is to provide a real-time vehicle violation detection device, which includes a processor and a memory coupled to the processor, wherein the memory stores There are computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the real-time vehicle violation detection method.
  • a technical solution adopted by this application is to provide a storage medium storing computer-readable instructions.
  • the computer-readable instructions are executed by one or more processors, one or Multiple processors execute the steps of the real-time vehicle violation detection method described above.
  • the real-time vehicle violation detection method, device, equipment, and storage device proposed in this application acquire video images in real time, and sequentially perform vehicle and lane line recognition on the i-th frame image and the i-1th frame image, and generate each identified vehicle Identification, and segmentation and matching of vehicles and lanes on the i-th frame image and the i-1th frame image according to the recognition result to obtain a vehicle segmentation matching image and a lane segmentation matching image, according to the vehicle segmentation matching image Match the image with the lane segmentation to determine whether there is a vehicle violation in the i-th frame image, and extract the violation vehicle information from the violation vehicle to achieve real-time automatic detection of vehicle violations and extraction of violation vehicle information.
  • the entire detection process does not require human intervention. The efficiency is high, and each frame is recognized, and there will be no missing frames, which avoids missed detection in the capture gap.
  • FIG. 1 is a schematic flowchart of a real-time vehicle violation detection method according to an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a vehicle matching process in an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • first”, “second”, and “third” in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined with “first”, “second”, and “third” may explicitly or implicitly include at least one of the features.
  • "a plurality of” means at least two, such as two, three, etc., unless otherwise specifically defined. All directional indications (such as up, down, left, right, front, back%) in the embodiments of this application are only used to explain the relative positional relationship between the components in a specific posture (as shown in the figure) , Movement status, etc., if the specific posture changes, the directional indication will also change accordingly.
  • FIG. 1 is a schematic flowchart of a real-time vehicle violation detection method according to an embodiment of the present application.
  • the real-time vehicle violation detection method can be run on a vehicle-mounted smart device, and the vehicle-mounted smart device can be installed on a vehicle (such as a bus, a traffic coordinator vehicle, a private car, etc.), and can obtain the front and rear lanes, The video image of the vehicle, and real-time detection of vehicle violations based on the video image.
  • This application can also be applied to smart transportation scenarios to promote the construction of smart cities. It should be noted that if there is substantially the same result, the method of the present application is not limited to the sequence of the process shown in FIG. 1. As shown in Figure 1, the method includes steps:
  • Step S11 Acquire a real-time video image.
  • the video image is captured by the vehicle-mounted smart device during driving.
  • the vehicle-mounted smart device may be a vehicle-mounted driving recorder and other photographing devices with driving shooting and recording functions.
  • Step S12 Cache the video image.
  • step S12 the video image is stored in real time.
  • the video image exceeds the storage capacity, at least part of the video image stored first may be deleted; in this embodiment, the at least part of the video image may be It is a frame of video image, that is, when the storage capacity is full, every time a frame of the video image at the current moment is newly stored, the earliest frame of the video image stored in the storage capacity is deleted.
  • Step S13 Perform vehicle and lane line recognition on the i-th frame image and the i-1th frame image in the video image, generate an identity mark for each recognized vehicle, and compare the i-th frame image and the image according to the recognition result.
  • the i-1th frame image performs segmentation and matching of vehicles and lanes to obtain vehicle segmentation matching images and lane segmentation matching images, where i is a natural number greater than or equal to 2.
  • step S13 includes the following steps:
  • Step S131 Identify each lane and its lane type, each vehicle and its vehicle attributes in each frame of image.
  • the vehicle attributes include one, two or more of color, model, brand, license plate, and the lane type includes real One, two or more of lanes, diversion lanes, and bus lanes.
  • step S131 image recognition technology can be used for each frame of image to recognize vehicles and lane lines as target objects.
  • vehicle attributes include one, two or more of color, model, brand, and license plate
  • the lane type includes one, two or more of solid lanes, diversion lanes, and bus lanes.
  • Step S132 Generate the identity of each vehicle according to the vehicle attributes.
  • Step S133 Segment each frame of image according to the recognition result to obtain a vehicle segmentation image and a lane segmentation image.
  • the vehicle and the lane line can be separately identified in each frame of image, and the vehicle and the lane line can be segmented from the image, and the vehicle segmentation image and the Lane segmentation image.
  • Step S134 Perform vehicle matching on the i-th frame image and the vehicle segmentation image of the i-1th frame image to obtain the vehicle segmentation matching image.
  • each vehicle in the vehicle segmentation image of the i-1th frame image is compared with each vehicle in the vehicle segmentation image of the i-th frame image to obtain a first similarity comparison result
  • the first similarity comparison result includes that each vehicle in the i-1th frame image and at least one vehicle in the i-th frame image are similar vehicles; in this embodiment, the i-1th frame image is
  • the similarity comparison between each vehicle in the vehicle segmentation image of the frame image and each vehicle in the vehicle segmentation image of the i-th frame image may be based on the vehicles in each frame of image identified in step S131 and their attributes of the vehicles, The first similarity comparison result is obtained according to the similarity of the vehicle attributes.
  • each vehicle in the i-1th frame image is randomly matched with one of the similar vehicles corresponding to it in the i-th frame image, and the result of random matching needs to satisfy
  • the vehicle of the image in the i-1th frame corresponds to the similar vehicle corresponding to it in the image of the i-th frame one-to-one, that is, multiple vehicles in the image of the i-1th frame cannot correspond to the i-th image at the same time.
  • the same vehicle in the image of the i-1th frame cannot simultaneously correspond to multiple of the similar vehicles in the image of the i-th frame, or the image of the i-1th frame
  • the remaining vehicles are matched with the unmatched vehicles in the i-th frame of image.
  • judging whether the remaining matching result is reasonable is judging whether the remaining vehicles in the i-1th frame of image are related to all the remaining vehicles. Whether the remaining matching results of the unmatched vehicles in the i-th image meet the first similarity comparison result, that is, the vehicle attributes of the remaining matching vehicles also satisfy the first similarity comparison result The similarity relationship.
  • the vehicle segmentation images of the i-th frame image and the i-1th frame image are spliced together and the matching vehicle is loaded with the matching line, thereby obtaining If the vehicle segmentation matching image is unreasonable, the vehicle in the i-1th frame image is matched with the other similar vehicle corresponding to it in the i-th frame image until the remaining matching If the result of is judged to be reasonable, the vehicle segmentation images of the i-th frame image and the i-1th frame image are spliced together, and the matching vehicle is loaded with a matching line, so as to obtain the vehicle segmentation matching image.
  • FIG. 2 is a schematic flowchart of a vehicle matching process in an embodiment of the present application.
  • Each vehicle in the vehicle segmentation image of the i-1th frame image is combined with the vehicle segmentation image of the i-th frame image Carry out similarity comparison for each vehicle in the vehicle.
  • the similarity comparison can be performed through the attributes of the vehicle to obtain a first similarity comparison result.
  • the first similarity comparison result includes the first similarity comparison result.
  • the x1 vehicle of the i-1 frame image is similar to the y1 vehicle and y2 vehicle of the i-th frame image, and the x2 vehicle of the i-1 frame image is similar to the y2 and y3 vehicles of the i-th frame image.
  • the x3 vehicle of the i-1th frame image is similar to the y1 vehicle of the current image, as shown in Figure 2(a).
  • the vehicles x1, x2, and x3 of the i-1th frame image are randomly matched with the corresponding similar vehicles of the i-th frame image, or the remaining vehicles of the i-1th frame image are matched with Perform residual matching on unmatched vehicles in the i-th frame of image, where the random matching result needs to satisfy the vehicle in the i-1th frame of image and the similar vehicles corresponding to it in the i-th frame of image one by one.
  • the x3 vehicle can only match the unmatched remaining vehicles y3 For vehicles, according to the first similarity comparison result x3, the remaining vehicles that are not matched by the vehicle y3 are not in the first similarity comparison result, and need to be matched again.
  • the The vehicle in the i-1th frame image is matched with the other similar vehicles corresponding to it in the i-th frame image until the result of the remaining matching is judged to be reasonable, as shown in Figure 2(b),
  • the matching process if the number of vehicles in the vehicle segmentation image of the i-1th frame image is different from the number of vehicles in the vehicle segmentation image of the i-th frame image, as in the i-th frame image
  • the vehicles in the vehicle segmentation image of the -1 frame image are x1, x2, x3, and the vehicles in the vehicle segmentation image of the i-th frame image are y1, y2, it can be considered that the vehicle in the i-1th frame image has one
  • the vehicle leaves the shooting screen it is only necessary to ensure the similarity comparison and matching of vehicles y1 and y2 in the vehicle segmentation image of the i-th frame image during the matching process; such as the vehicle segmentation image of the i-1th frame image
  • the vehicles in the i-th frame are x1, x2, and the vehicles in the vehicle segmentation image of the i-th frame are y1, y2, and y3.
  • Step S135 Perform lane matching on the i-th frame image and the lane division image of the i-1th frame image to obtain the lane division matching image.
  • the acquisition process of the lane segmentation matching image is similar to the acquisition process of the vehicle segmentation matching image in step S134, that is, each lane in the lane segmentation image of the i-1th frame image and the lane of the i-th frame image Perform similarity comparison for each lane in the segmented image to obtain a second similarity comparison result.
  • the second similarity comparison result includes each lane of the i-1th frame image and the i-th frame image At least one lane of is a similar lane; in this embodiment, each lane in the lane segmentation image of the i-1 frame image and each lane in the lane segmentation image of the i-th frame image are compared for similarity
  • the second similarity comparison result may be obtained according to the lane types and the lane types in each frame of image identified in step S131.
  • each lane of the i-1th frame image is randomly matched with one of the similar lanes corresponding to it in the i-th frame image, and the result of random matching needs to satisfy the
  • the lanes of the i-1 frame image correspond to the similar lanes corresponding to it in the i frame image, that is, the multiple lanes of the i-1 frame image cannot correspond to the i frame image at the same time
  • the same lane in the i-1th frame image cannot correspond to multiple similar lanes in the i-th frame image at the same time, or the remaining lanes of the i-1th frame image Perform residual matching with the unmatched lane in the i-th frame of image.
  • judging whether the remaining matching result is reasonable is to judge whether the remaining lanes of the i-1th frame image and the remaining lanes are reasonable. Whether the remaining matching results of the unmatched lanes in the i-th frame image satisfy the second similarity comparison result, that is, the lane types of the remaining matching lanes also satisfy the second similarity comparison result The similarity relationship.
  • the lane segmentation images of the i-th frame image and the i-1th frame image are stitched together and the matching lanes are loaded with matching lines to obtain If the lane segmentation matching image is unreasonable, the lane of the i-1 frame image is matched with the other similar lane corresponding to it in the i frame image until the remaining matching lane If the result is judged to be reasonable, the lane segmentation images of the i-th frame image and the i-1th frame image are spliced together, and the matching lanes are loaded with matching lines, so as to obtain the lane segmentation matching image.
  • the following also uses an example to illustrate the above lane matching process.
  • the similarity comparison between each lane in the lane segmentation image of the i-1th frame image and each lane in the lane segmentation image of the i-th frame image is performed to obtain a second similarity comparison result.
  • the two similarity comparison results include that the lane m1 of the image of the i-1th frame is similar to the lanes n1 and n2 of the image of the i-th frame, and the lane m2 of the image of the i-1th frame is similar to that of the i-th frame.
  • the n2 lane and the n3 lane of the image are similar, and the m3 lane of the i-1th frame image is similar to the n1 lane of the current image.
  • the matching process if the number of lanes in the lane division image of the i-1 frame image is different from the number of lanes in the lane division image of the i frame image, as in the i-1 frame image
  • the lanes in the lane segmentation image of the image are m1, m2, m3, and the lanes in the lane segmentation image of the i-th frame image are n1, n2, then it can be considered that the lanes in the lane segmentation image of the i-th frame image are
  • the matching process only needs to ensure the similarity comparison and matching of the lanes n1 and n2 in the lane segmentation image of the i-th frame image; as in the lane segmentation image of the i-1th frame image
  • the lanes in the ith frame are m1 and m2, and the lanes in the lane segmentation image of the i-th frame image are n1, n2, and n3.
  • the lane in the lane segmentation image of the i-th frame image has a new lane.
  • the matching process it is only necessary to ensure the similarity comparison and matching of lanes m1 and m2 in the lane segmentation image of the i-1th frame image, and the unmatched lane in the lane segmentation image of the i-th frame image can be in the next
  • the frame is the lane segmentation image of the i+1th frame image to compare and match.
  • Step S14 Determine whether there is a vehicle violation in the i-th frame image according to the vehicle segmentation matching image and the lane segmentation matching image; if it is determined that there is a vehicle violation, step S15 is executed.
  • step S14 judging whether there is a vehicle violation in the i-th frame image may be based on the lane attributes of each lane in the lane segmentation matching image and combining with the vehicle matching line in the vehicle segmentation matching image to determine whether there is a vehicle violation. If the lane segmentation matching image has a lane with preset lane attributes, and there is a vehicle matching line extending to the lane with preset lane attributes and other lanes, then it is determined that the vehicle corresponding to the vehicle matching line is an offending vehicle .
  • the preset lane attributes include one, two or more of a solid lane, a diversion lane, and a bus lane.
  • the lane attributes of the lane in the lane segmentation matching image are identified with the above-mentioned preset lane attributes, it is determined whether the vehicle violates the rules by detecting whether the vehicle matching line of the vehicle extends to the lane with the preset lane attributes and other lanes.
  • the following is for solid lanes and diversion lines. Lanes and bus lanes are described.
  • the vehicle matching line of a vehicle is from the solid lane If one lane extends to the other lane, it can be considered that the vehicle changes lanes across a solid line, causing a violation;
  • the lane attribute of the lane in the lane segmentation matching image is a diversion lane, if there is If the vehicle matching line of the vehicle extends from the outer lane of the diversion lane to the inner side of the diversion lane, it can be considered that the vehicle occupies the diversion lane, causing a violation; when the lane segmentation matching image is identified
  • the lane attribute of the lane is a bus lane
  • the vehicle matching line of a vehicle extends from the left lane of the bus lane to the right side of the bus lane, it can be considered that this vehicle occupies the bus lane, resulting in Violation.
  • Step S15 Extract the vehicle information of the offending vehicle according to the i-th frame image.
  • the vehicle information of the violation vehicle can be extracted according to the i-th frame image.
  • the vehicle information of the violating vehicle may be the license plate number of the violating vehicle.
  • Step S16 It is determined whether the extraction of the vehicle information of the violating vehicle is successful, if the extraction of the vehicle information of the violating vehicle is successful, step S17 is executed, and if the extraction of the vehicle information of the violating vehicle fails, then the step S18 is executed.
  • the vehicle information of the vehicle in the i-th frame of the vehicle may be blurred or concealed. Extracting the vehicle information of the offending vehicle from the i-th frame image. Therefore, it is necessary to first determine whether the extraction of the vehicle information of the offending vehicle is successful.
  • Step S17 Send the extracted vehicle information of the illegal vehicle and the i-th frame image or the vehicle information, the i-th frame image and the i-1th frame image to report the violation.
  • the vehicle information involved in the violation report and the i-th frame image or the vehicle information, the i-th frame image and the i-1th frame image are stored ; And every predetermined time or when the preset conditions are met, the stored vehicle information and the i-th frame image or the vehicle information and the i-th frame image and the i-th frame involved in the reported violation
  • One frame of image is stored and uploaded to a remote server, and after the upload is completed, the stored vehicle information and the i-th frame of image or the vehicle information and the i-th frame of image and all the images involved in the violation report are deleted.
  • the i-1th frame image is stored and uploaded to a remote server, and after the upload is completed, the stored vehicle information and the i-th frame of image or the vehicle information and the i-th frame of image and all the images involved in the violation report are deleted.
  • the predetermined time can be set as a time when the system is relatively idle, such as uploading cached violation records at night; it can also be set as a preset condition that the cached violation records are uploaded when a vehicle violation is not detected for a period of time.
  • Step S18 Obtain the identity of the offending vehicle; and obtain corresponding vehicle information in the cached video image according to the identity, and send the corresponding vehicle information and the i-th frame image or the vehicle Information and the i-th frame image and the i-1th frame image are reported in violation of regulations.
  • the vehicle information may be a license plate number.
  • the vehicle information involved in the violation report and the i-th frame image or the vehicle information the vehicle information involved in the violation report and the i-th frame image or the vehicle information
  • the i-th frame image and the i-1th frame image can be performed Storing; and storing the stored vehicle information and the i-th frame image or the vehicle information and the i-th frame image and the i-th frame image and the i-th frame image involved in the violation report every predetermined time or when a preset condition is met -1 frame image is stored and uploaded to a remote server, and after the upload is completed, the stored vehicle information and the i-th frame image involved in the violation report or the vehicle information and the i-th frame image and The i-1th frame image.
  • the real-time vehicle violation detection method of an embodiment of the present application acquires video images in real time, and sequentially recognizes vehicles and lane lines on the i-th frame image and the i-1th frame image, and generates an identity mark for each identified vehicle, and based on According to the recognition result, segment and match the i-th frame image and the i-1th frame image of vehicles and lanes to obtain a vehicle segmentation matching image and a lane segmentation matching image, according to the vehicle segmentation matching image and the lane segmentation
  • the matching image determines whether there is a vehicle violation in the i-th frame image, and extracts the violation vehicle information from the violation vehicle to realize the real-time automatic detection of vehicle violations and the extraction of the violation vehicle information.
  • the entire detection process does not require human intervention, and the detection efficiency is high, and Every frame is recognized, and no missing frames will occur, avoiding missed inspections in the capture gap.
  • the corresponding vehicle information is obtained from the cached video image through the identity of the illegal vehicle, which ensures the accuracy and effectiveness of the extraction of the illegal vehicle information.
  • the i-th frame image and the i-1th frame image are identified in turn for vehicles and lane lines, and the identification of each vehicle is generated according to the vehicle attributes for each recognized vehicle, so that each vehicle in the video image The ID is unique.
  • each vehicle in the vehicle segmentation image of the i-1th frame image, each lane line in the lane line segmentation image, and each vehicle and lane line segmentation image in the vehicle segmentation image of the i-th frame image has a simple algorithm, high operating efficiency, and can quickly and accurately match vehicles and lane lines.
  • the vehicle can be completed by one frame of image or two consecutive frames of images. Automatic detection of violations has high detection efficiency and accurate detection results.
  • the video image exceeds the storage capacity, at least part of the video image stored first is deleted, and the video image stored first is deleted, without affecting the real-time violation detection result, but also freeing up storage space.
  • storing the vehicle information involved in the violation report and the i-th frame image or the i-th frame image and the i-1th frame image can effectively save the violation evidence.
  • the stored vehicle information involved in the violation report and the i-th frame image or the i-th frame image and the i-1th frame image are stored every predetermined time or when a preset condition is met Upload to a remote server and delete the local storage after the upload is completed, which can complete the upload of the violation evidence when the system is idle, guarantees the upload success rate, and deletes the local storage after the upload is successful, and further releases the local storage space.
  • Fig. 3 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the present application.
  • the real-time vehicle violation detection device 20 includes a video receiving module 21, a video caching module 22, an information extraction module 23, a violation judgment module 24, a violation information extraction module 25, a violation information storage module 26, and a violation information transmission module. Module 27.
  • the video receiving module 21 is configured to obtain real-time video images; in this embodiment, the video receiving module 21 can receive video images captured by a vehicle-mounted camera during driving.
  • the video buffer module 22 is configured to buffer the video image acquired by the video receiving module 21; when the video image exceeds the storage capacity of the video buffer module 22, at least part of the video stored first can be deleted image.
  • the information extraction module 23 is used to identify vehicles and lane lines in the i-th frame image and the i-1th frame image in the video image acquired by the video receiving module 21, and generate an identity mark for each identified vehicle, And according to the recognition result, segmentation and matching of vehicles and lanes are performed on the i-th frame image and the i-1th frame image to obtain a vehicle segmentation matching image and a lane segmentation matching image, where i is a natural number greater than or equal to 2, so
  • the information extraction module 23 can use image recognition technology for each frame of image to recognize vehicles and lane lines as target objects. Through recognition, each frame of image and its lane type, each vehicle and its vehicle attributes are obtained.
  • the vehicle attributes include one, two or more of color, model, brand, and license plate
  • the lane type includes one, two or more of solid lanes, diversion lanes, and bus lanes.
  • the identity of each vehicle is generated according to the attributes of the vehicle.
  • the information extraction module 23 can respectively identify the vehicle and the lane line in each frame of image, and segment the vehicle and the lane line from the image, and obtain the respective The vehicle segmentation image and the lane segmentation image.
  • each vehicle in the vehicle segmentation image of the i-1th frame image is compared with each vehicle in the vehicle segmentation image of the i-th frame image to obtain a first similarity comparison result
  • the first similarity comparison result includes that each vehicle in the i-1th frame image and at least one vehicle in the i-th frame image are similar vehicles; A vehicle is randomly matched with one of the similar vehicles corresponding to it in the i-th frame of image or residual matching is performed with an unmatched vehicle in the i-th frame of image;
  • the vehicle segmentation images of the i-th frame image and the i-1th frame image are spliced together and the matched
  • the vehicle loads the matching line to obtain the vehicle segmentation matching image. If it is unreasonable, the vehicle in the i-1th frame image and the other similar vehicle corresponding to it in the i-th frame image are performed Matching, until the result of the remaining matching is judged to be reasonable, the vehicle segmentation images of the i-th frame image and the i-1th frame image are spliced together and the matching vehicle is loaded with the matching line, so as to obtain the Vehicle segmentation matches the image.
  • the process of acquiring the lane segmentation matching image by the information extraction module 23 is similar to the foregoing process of acquiring the vehicle segmentation matching image, that is, each lane in the lane segmentation image of the i-1th frame image and the i-th
  • the similarity comparison of each lane in the lane segmentation image of the frame image is performed to obtain a second similarity comparison result.
  • the second similarity comparison result includes each lane of the i-1th frame image and the At least one lane of the i-th frame image is a similar lane; each lane of the i-1th frame image is randomly matched with one of the similar lanes corresponding to it in the i-th frame image or is matched with all lanes.
  • the unmatched lanes in the i-th frame of image are subjected to residual matching,
  • the second similarity comparison result it is judged whether the remaining matching result is reasonable; if it is reasonable, the lane segmentation images of the i-th frame image and the i-1th frame image are stitched together and the matched Load a matching line into the lane to obtain the lane segmentation matching image. If it is unreasonable, match the lane of the i-1 frame image with the other similar lane corresponding to it in the i frame image Until the result of the remaining matching is judged to be reasonable, the lane segmentation images of the i-th frame image and the i-1th frame image are stitched together and the matching lane is loaded with the matching line, thereby obtaining the lane Segment matching images.
  • the violation judgment module 24 is used for judging whether there is a vehicle violation in the i-th frame image according to the vehicle segmentation matching image and the lane segmentation matching image extracted by the information extraction module 23; when it is determined that there is a vehicle violation, based on The i-th frame image extracts the vehicle information of the vehicle in violation. Specifically, the violation judgment module 24 judges whether there is a vehicle violation in the i-th frame image can be based on the lane segmentation matching image of each lane The lane attributes are combined with the vehicle matching line in the vehicle segmentation matching image to determine whether there is a vehicle violation.
  • the vehicle information of the violation vehicle can be extracted according to the i-th frame image.
  • the vehicle information of the violating vehicle may be the license plate number of the vehicle.
  • the violation judgment module 24 succeeds in extracting the vehicle information of the violation vehicle, the vehicle information involved in the violation report and the i-th frame image or the vehicle information and the i-th frame image and the first
  • the i-1 frame image is sent to the violation information storage module 26 for storage; the violation information transmission module 27 collects the stored violation information from the violation information storage module 26 at predetermined intervals or when preset conditions are met.
  • the vehicle information involved in the violation report and the i-th frame image or the vehicle information, the i-th frame image and the i-1th frame image are stored and uploaded to the remote server, and the said uploading is completed.
  • the violation information storage module 26 stores the vehicle information and the i-th frame image or the vehicle information, the i-th frame image, and the i-1th frame image involved in the violation report.
  • the identity identifier of the violation vehicle is sent to the violation information extraction module 25, and the violation information extraction module 25 is based on the identity identifier of the violation vehicle.
  • the corresponding vehicle information is obtained from the video images buffered by the video receiving module 21.
  • the vehicle information may be a license plate number.
  • the violation information extraction module 25 obtains the vehicle information, it can report the vehicle information involved in the violation report and the i-th frame image or the vehicle information and the i-th frame image and the The i-1 frame image is sent to the violation information storage module 26 for storage; the violation information sending module 27 collects the stored information from the violation information storage module 26 at predetermined intervals or when preset conditions are met.
  • the vehicle information involved in the violation report and the i-th frame image or the vehicle information, the i-th frame image and the i-1th frame image are stored and uploaded to the remote server, and all the images are deleted after the upload is completed.
  • FIG. 4 is a schematic structural diagram of a real-time vehicle violation detection device according to an embodiment of the present application.
  • the real-time vehicle violation detection device 30 includes a memory 32, a processor 31, and a computer program stored on the memory 32 and running on the processor 31, and the processor 31 executes all The computer program implements the real-time vehicle violation detection method.
  • FIG. 5 is a schematic structural diagram of a storage medium according to an embodiment of the present application.
  • a storage medium storing computer-readable instructions 41, when the computer-readable instructions 41 are executed by one or more processors, cause one or more processors to execute the real-time vehicle violation detection method.
  • the computer program can be stored in a computer readable storage medium. When executed, it may include the procedures of the above-mentioned method embodiments.
  • the aforementioned storage medium may be a magnetic disk, an optical disk, or a read-only storage memory (Read-Only Memory, ROM) and other non-volatile storage media, or random storage memory (Random Access Memory, RAM) and other volatile storage media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种实时车辆违章检测方法、装置、设备及存储介质。该方法包括：获取实时视频图像并缓存；对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别车辆生成身份标识，依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道分割与匹配，获得车辆分割匹配图像、车道分割匹配图像；依据所述车辆分割匹配图像与所述车道分割匹配图像判断否有违章(S14)；若判断有违章，则依据所述第i帧图像进行车辆信息提取、违章上报；若车辆信息提取失败，则获取所述违章车辆的身份标识在缓存所述视频图像中获取车辆信息、违章上报。该方法能达到实时自动违章检测，且可应用于智慧交通场景中，从而推动智慧城市的建设的目的。

Description

实时车辆违章检测方法、装置、设备及存储介质
本申请要求于2020年05月28日提交中国专利局、申请号为202010470098. 2，发明名称为“实时车辆违章检测方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理技术领域，尤其涉及一种实时车辆违章检测方法、装置、设备及存储介质。
 背景技术
交通违章智能检测是使用各类传感器和影像获取设备，结合后端的违章判断算法和信息提取算法，自动对车辆进行违章判断。这类技术可以帮助交通管理部门在提高违章判断速度、准确度的同时，降低人工成本，避免人为原因造成的误检、漏检等偏差。
目前市面上的智能违章检测技术分为两种：一种是将违章检测的硬件装置安装在固定位置，使用光线或压力等传感器感应车辆的位置等信息，再结合拍照获取的单张或若干张照片来做违章判断和信息提取。发明人意识到此种违章检测的硬件装置只能安装在固定的位置，来检测特定的违章，不够灵活，只能在固定位置对车辆进行监督。
另一种是通过车载摄像头获取行车视频，再对视频中的一帧或某几帧在本地进行处理，或发送到远程服务器进行处理进行违章判断。发明人发现对视频中的一帧或某几帧在本地进行处理可能造成车辆信息提取困难，如提取的图像中车牌被遮挡等；发送到远程服务器进行处理加重了服务器下载负担，且无法大规模使用。
 技术问题
本申请提供一种实时车辆违章检测方法、装置、设备及存储装置，能够达到实时自动违章检测，且精准提取违章车辆信息的目的。
 技术解决方案
为解决上述技术问题，本申请采用的一个技术方案是：提供一种实时车辆违章检测方法，包括以下步骤：
获取实时视频图像；
将所述视频图像进行缓存；
对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数；
依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；
若判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取；
若所述车辆信息提取成功，则发送提取的所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报；
若所述车辆信息提取失败，则获取所述违章车辆的身份标识；及
依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息，发送所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
此外，为解决上述技术问题，本申请还采用的一个技术方案是：提供一种实时车辆违章检测装置，该装置包括：
视频接收模块，用于获取实时视频图像；
视频缓存模块，用于将所述视频图像进行缓存；
信息提取模块，用于对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数；
违章判断模块，用于依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；当判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取；
违章信息提取模块，用于当所述违章判断模块提取所述违章车辆的车辆信息提取失败时，获取所述违章车辆的身份标识；及依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息；
违章信息存储模块，用于对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行存储；
违章信息发送模块，用于发送所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
此外，为解决上述技术问题，本申请还采用的一个技术方案是：提供一种实时车辆违章检测设备，该设备包括处理器、与所述处理器耦接的存储器，其中，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行上述所述实时车辆违章检测方法的步骤。
此外，为解决上述技术问题，本申请还采用的一个技术方案是：提供一种存储有计算机可读指令的存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述所述实时车辆违章检测方法的步骤。
 有益效果
本申请提出的实时车辆违章检测方法、装置、设备及存储装置，通过实时获取视频图像，并依次对第i帧图像及第i-1帧图像进行车辆、车道线识别并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章，并对违章车辆提取违章车辆信息，实现车辆违章的实时自动检测及违章车辆信息提取，整个检测过程无需人工介入，检测效率高，并且对每一帧进行识别，不会出现漏帧，避免了抓拍间隙中的漏检。
 附图说明
图1是本申请一种实施例的实时车辆违章检测方法的流程示意图；
图2是本申请一种实施例中车辆匹配过程的流程示意图；
图3是本申请一种实施例的实时车辆违章检测装置的结构示意图；
图4是本申请一种实施例的实时车辆违章检测设备的结构示意图；
图5是本申请一种实施例的存储介质的结构示意图。
 本发明的实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请中的术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”、“第三”的特征可以明示或者隐含地包括至少一个该特征。本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。本申请实施例中所有方向性指示（诸如上、下、左、右、前、后……）仅用于解释在某一特定姿态（如附图所示）下各部件之间的相对位置关系、运动情况等，如果该特定姿态发生改变时，则该方向性指示也相应地随之改变。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
图1是本申请一种实施例的实时车辆违章检测方法的流程示意图。所述实时车辆违章检测方法可以运行于一车载智能设备上，所述车载智能设备可以安装在一交通工具（如公交车、交通协管车辆、私家车等）上，可以获取前后方的车道、车辆的视频图像，并依据所述视频图像实时对车辆违章进行检测检测。本申请还可应用于智慧交通场景中，从而推动智慧城市的建设。需注意的是，若有实质上相同的结果，本申请的方法并不以图1所示的流程顺序为限。如图1所示，该方法包括步骤：
步骤S11：获取实时视频图像。
需要说明的是，本实施例中，所述视频图像是通过所述车载智能设备在行车过程中拍摄获得，所述车载智能设备可以是车载行车记录仪等具有行车拍摄记录功能的拍摄装置。
步骤S12：将所述视频图像进行缓存。
步骤S12中，将所述视频图像实时进行存储，当所述视频图像超过存储容量时可以删除最先存储的至少部分所述视频图像；在本实施例中，所述至少部分所述视频图像可以为一帧视频图像，即当存储容量存满时，每新存入一帧当前时刻的所述视频图像，则删除所述存储容量中已存储的最早的一帧所述视频图像。
步骤S13：对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数。
具体地，步骤S13包括以下步骤：
步骤S131：识别每帧图像中的各车道及其车道类型、各车辆及其车辆属性，所述车辆属性包括颜色、车型、品牌、车牌中一个、两个或多个，所述车道类型包括实线道、导流线道、公交车道中的一个、两个或多个。
可以理解，步骤S131中，可以对每帧图像采用图像识别技术将车辆、车道线作为目标对象分别进行识别，通过识别获取每帧图像中的各车道及其车道类型、各车辆及其车辆属性，本实施例中，所述车辆属性包括颜色、车型、品牌、车牌中一个、两个或多个，所述车道类型包括实线道、导流线道、公交车道中的一个、两个或多个。
步骤S132：依据所述车辆属性生成各车辆的身份标识。
根据本申请的一种实施例，可以依据所述车辆属性生成各车辆的身份标识，如对所述车辆属性中的各属性设置属性值和权重，通过所述属性值和所述权重计算所述身份标识，例如：一辆车的所述车辆属性为红色、小轿车，设置“红色”的属性值为10，权重为0.4，设置“小轿车”的属性值为20，权重为0.6，则可以依据属性值，权重为所述车辆生成身份标识10*0.4+20*0.6=16，以上为一种实施例的举例说明，对于不同的所述车辆属性还可以采用其他类似的计算方法，此处不再一一举例。
步骤S133：依据所述识别结果分割每帧图像获得车辆分割图像与车道分割图像。
通过图像识别技术，可以在每帧图像中分别识别出所述车辆、所述车道线，并将所述车辆、所述车道线从图像中分割出来，分别获取到所述车辆分割图像与所述车道分割图像。
步骤S134：对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像。
具体的，将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对，获得第一相似性比对结果，所述第一相似性比对结果包括所述第i-1帧图像的每一车辆与所述第i帧图像的至少一辆车辆为相似车辆；本实施例中，将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对可以依据步骤S131中识别的每帧图像中各车辆及其所述车辆属性，依据所述车辆属性的相似性获得第一相似性比对结果。
将所述第i-1帧图像的每一车辆与所述第i帧图像中与其相对应的所述相似车辆中的一辆进行随机匹配或与所述第i帧图像中未匹配车辆进行剩余匹配；本实施例中，将所述第i-1帧图像的每一车辆与所述第i帧图像中与其相对应的所述相似车辆中的一辆进行随机匹配，随机匹配的结果需要满足所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的所述相似车辆一一对应，即所述第i-1帧图像的多辆车辆不能同时对应所述第i帧图像中同一辆所述相似车辆，所述第i-1帧图像的同一车辆也不能同时对应所述第i帧图像中多辆所述相似车辆，或对所述第i-1帧图像的剩余车辆与所述第i帧图像中未匹配车辆进行剩余匹配。
进一步地，依据所述第一相似性比对结果判断所述剩余匹配结果是否合理，本实施例中，判断所述剩余匹配结果是否合理即判断所述第i-1帧图像的剩余车辆与所述第i帧图像中未匹配车辆的剩余匹配结果是否满足所述第一相似性比对结果，也就是说，剩余匹配的车辆的所述车辆属性也满足所述第一相似性比对结果中的相似性关系。若合理，即满足所述第一相似性比对结果，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像，若不合理，则将所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的其他一辆所述相似车辆进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像。
下面以一个例子来说明上述车辆匹配过程。请参阅图2，图2是本申请一种实施例中车辆匹配过程的流程示意图，将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对，本实施例中，可以通过所述车辆属性进行相似性比对，从而获得第一相似性比对结果，所述第一相似性比对结果包括所述第i-1帧图像的x1车辆与所述第i帧图像的y1车辆及y2车辆相似、所述第i-1帧图像的x2车辆与所述第i帧图像的y2车辆及y3车辆相似、所述第i-1帧图像的x3车辆与所述当前图像的y1车辆相似，如图2（a）所示。
进一步地，将所述第i-1帧图像的x1、x2、x3车辆与所述第i帧图像的对应的所述相似车辆进行随机匹配或对所述第i-1帧图像的剩余车辆与所述第i帧图像中未匹配车辆进行剩余匹配，其中，随机匹配的结果需要满足所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的所述相似车辆一一对应，若x1车辆随机匹配与其对应的所述相似车辆中未匹配的y1车辆， x2车辆匹配与其对应的所述相似车辆中未匹配的y2车辆，则x3车辆只能匹配未匹配的剩余车辆y3车辆，依据所述第一相似性比对结果x3车辆匹配未匹配的剩余车辆y3车辆不在所述第一相似性比对结果中，则需要进行再次匹配，本举例中，再次匹配时将所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的其他一辆所述相似车辆进行匹配，直到所述剩余匹配的结果判断为合理，如图2（b）所示，将x1车辆与y1与y2的另外一车辆匹配，如x1车辆匹配y2车辆，将x2与余下未匹配的且与x2相似的另一车辆匹配，如x2车辆匹配y3车辆，将x3与余下未匹配的且与x3相似的y1车辆匹配，即完成了所述第i帧图像中每一车辆与所述第i-1帧图像的车辆的相似匹配，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像。
需要说明的是，在匹配过程中，若所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆数量不同，如所述第i-1帧图像的车辆分割图像中的车辆为x1、x2、x3，所述第i帧图像的车辆分割图像中的车辆为y1、y2，则可以认为所述第i-1帧图像的车辆有一辆离开拍摄画面，则在匹配过程中只需保证所述第i帧图像的车辆分割图像中的车辆y1、y2的相似性比对、匹配；如所述第i-1帧图像的车辆分割图像中的车辆为x1、x2，所述第i帧图像的车辆分割图像中的车辆为y1、y2、y3，则可以认为所述第i帧图像的车辆有一辆进入拍摄画面，则在匹配过程中只需保证所述第i-1帧图像的车辆分割图像中的车辆x1、x2的相似性比对、匹配，所述第i帧图像的车辆分割图像中的未匹配车辆可以在下一帧即第i+1帧图像的车辆分割图像中的车辆中去比对和匹配。
步骤S135：对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像。
所述车道分割匹配图像获取过程与上述步骤S134中所述车辆分割匹配图像的获取过程类似，即将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对，获得第二相似性比对结果，所述第二相似性比对结果包括所述第i-1帧图像的每一车道与所述第i帧图像的至少一条车道为相似车道；本实施例中，将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对可以依据步骤S131中识别的每帧图像中各车道及其所述车道类型，依据所述车道类型获得第二相似性比对结果。
将所述第i-1帧图像的每一车道与所述第i帧图像中与其相对应的所述相似车道中的一条进行随机匹配或与所述第i帧图像中未匹配车道进行剩余匹配；本实施例中，将所述第i-1帧图像的每一车道与所述第i帧图像中与其相对应的所述相似车道中的一条进行随机匹配，随机匹配的结果需要满足所述第i-1帧图像的车道与所述第i帧图像中与其相对应的所述相似车道一一对应，即所述第i-1帧图像的多条车道不能同时对应所述第i帧图像中同一条所述相似车道，所述第i-1帧图像的同一车道也不能同时对应所述第i帧图像中多条所述相似车道，或对所述第i-1帧图像的剩余车道与所述第i帧图像中未匹配车道进行剩余匹配。
进一步地，依据所述第二相似性比对结果判断所述剩余匹配结果是否合理，本实施例中，判断所述剩余匹配结果是否合理即判断所述第i-1帧图像的剩余车道与所述第i帧图像中未匹配车道的剩余匹配结果是否满足所述第二相似性比对结果，也就是说，剩余匹配的车道的所述车道类型也满足所述第二相似性比对结果中的相似性关系。若合理，即满足所述第二相似性比对结果，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像，若不合理，则将所述第i-1帧图像的车道与所述第i帧图像中与其相对应的其他一条所述相似车道进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像。
下面同样以一个例子来说明上述车道匹配过程。将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对，获得第二相似性比对结果，所述第二相似性比对结果包括所述第i-1帧图像的m1车道与所述第i帧图像的n1车道及n2车道相似、所述第i-1帧图像的m2车道与所述第i帧图像的n2车道及n3车道相似、所述第i-1帧图像的m3车道与所述当前图像的n1车道相似。
将所述第i-1帧图像的m1、m2、m3车道与所述第i帧图像的对应的所述相似车道进行随机匹配或对所述第i-1帧图像的剩余车道与所述第i帧图像中未匹配车道进行剩余匹配，如将m1车道与n1车道匹配，将m2车道与n2车道匹配， m3与余下的n3车道匹配，依据所述第二相似性比对结果m3车道匹配n3车道不在所述第二相似性比对结果中，则需要进行再次匹配，本举例中，再次匹配时，可以将m1车道与n2车道匹配，将m2车道与n3车道匹配，将m3车道与n1车道匹配，即完成了所述第i帧图像中每一车道与所述第i-1帧图像的车道的相似匹配，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像。
同样，在匹配过程中，若所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道数量不同，如所述第i-1帧图像的车道分割图像中的车道为m1、m2、m3，所述第i帧图像的车道分割图像中的车道为n1、n2，则可以认为所述第i帧图像的车道分割图像中的车道有一道行程终止，则在匹配过程中只需保证所述第i帧图像的车道分割图像中的各车道n1、n2的相似性比对、匹配；如所述第i-1帧图像的车道分割图像中的车道为m1、m2，所述第i帧图像的车道分割图像中的车道为n1、n2、n3，则可以认为所述第i帧图像的车道分割图像中的车道有一新车道，则在匹配过程中只需保证所述第i-1帧图像的车道分割图像中的车道m1、m2的相似性比对、匹配，所述第i帧图像的车道分割图像中的未匹配车道可以在下一帧即第i+1帧图像的车道分割图像中的车道中去比对和匹配。
步骤S14：依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；若判断有车辆违章，则执行步骤S15。
可以理解，依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中没有车辆违章，则可以继续对后续第i+1帧图像采用同样的方法进行违章判断。
步骤S14中，对所述第i帧图像中是否有车辆违章进行判断可以依据所述车道分割匹配图像中的各车道的车道属性并结合所述车辆分割匹配图像中车辆匹配线判断是否有车辆违章，若所述车道分割匹配图像中具有预设车道属性的车道，且有车辆匹配线延伸至所述具有预设车道属性的车道与其他车道，则判断所述车辆匹配线对应的车辆是违章车辆。
本实施例中，所述预设车道属性包括实线道、导流线道、公交车道中的一个、两个或多个，当识别到所述车道分割匹配图像中的车道的所述车道属性具有上述预设车道属性时，通过检测车辆的所述车辆匹配线是否延伸至所述具有预设车道属性的车道与其他车道来判断所述车辆是否违章，下面分别针对实线道、导流线道、公交车道三种车道进行说明，当识别到所述车道分割匹配图像中的车道的所述车道属性为实线道时，若有车辆的所述车辆匹配线从所述实线道的一侧车道延伸至另一侧车道，则可以认为此车辆跨实线变道，造成违章；当识别到所述车道分割匹配图像中的车道的所述车道属性为导流线道时，若有车辆的所述车辆匹配线从所述导流线道的外侧车道延伸至导流线道内侧，则可以认为此车辆占用导流线道，造成违章；当识别到所述车道分割匹配图像中的车道的所述车道属性为公交车道时，若有车辆的所述车辆匹配线从所述公交车道的左侧车道延伸至公交车道右侧，则可以认为此车辆占用公交车道，造成违章。
步骤S15：依据所述第i帧图像进行违章车辆的车辆信息提取。
当检测到有车辆违章时，可以依据所述第i帧图像进行违章车辆的车辆信息提取。本实施例中，所述违章车辆的车辆信息可以为违章车辆的车牌号。
步骤S16：判断所述违章车辆的车辆信息提取是否成功，若所述违章车辆的车辆信息提取成功，则执行步骤S17，若所述违章车辆的车辆信息提取失败，则执行步骤S18。
需要说明的是，由于车辆违章的时候由于某些原因，例如距离太远或有障碍物遮挡等，可能使得所述第i帧图像中车辆的所述违章车辆的车辆信息模糊或被遮掩，无法从所述第i帧图像中提取所述违章车辆的车辆信息。因此，需要先判断所述违章车辆的车辆信息提取是否成功。
步骤S17：发送提取的所述违章车辆的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
若所述违章车辆的车辆信息提取成功，对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行存储；及每隔预定时间或满足预设条件时将所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像存储上传到远程服务器，并在所述上传完成后删除所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像。本实施例中，预定时间可以设置为系统较为空闲的时间，如夜间上传缓存的违章记录；也可以设置预设条件为一段时间间隔没有检测到车辆违章时上传缓存的违章记录。
步骤S18：获取所述违章车辆的身份标识；及依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息，发送所述对应的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
若所述违章车辆的车辆信息提取失败，则需要通过违章车辆的所述身份标识在缓存的所述视频图像中获取对应的车辆信息，本实施例中，所述车辆信息可以是车牌号。同样，在获取到所述车辆信息后，可以对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行存储；及每隔预定时间或满足预设条件时将所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像存储上传到远程服务器，并在所述上传完成后删除所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像。
本申请一种实施例的实时车辆违章检测方法通过实时获取视频图像，并依次对第i帧图像及第i-1帧图像进行车辆、车道线识别并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章，并对违章车辆提取违章车辆信息，实现车辆违章的实时自动检测及违章车辆信息提取，整个检测过程无需人工介入，检测效率高，并且对每一帧进行识别，不会出现漏帧，避免了抓拍间隙中的漏检。
进一步地，当违章车辆信息提取失败时，通过所述违章车辆的身份标识在缓存的所述视频图像中获取对应的车辆信息，保障了违章车辆信息提取的准确性、有效性。
进一步地，依次对第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆依据所述车辆属性生成各车辆的身份标识，使所述视频图像中每辆车的所述身份标识唯一。
进一步地，将所述第i-1帧图像的车辆分割图像中的各车辆、车道线分割图像中的各车道线与所述第i帧图像的车辆分割图像中的各车辆、车道线分割图像中的各车道线进行相似性比对获取所述车辆分割匹配图像、所述车道线分割匹配图像，算法简单，运行效率高，能快速、准确对车辆、车道线进行匹配。
进一步地，依据所述车道分割匹配图像中的各车道的车道属性并结合所述车辆分割匹配图像中车辆匹配线判断是否有车辆违章，可以实现通过一帧图像或者连续两帧图像即可完成车辆违章的自动检测，检测效率高，检测结果准确。
进一步地，当所述视频图像超过存储容量时删除最先存储的至少部分所述视频图像，删除最先存储的视频图像，不影响实时违章检测结果，还能释放存储空间。
进一步地，对所述违章上报涉及的车辆信息及所述第i帧图像或所述第i帧图像及所述第i-1帧图像进行存储，能有效的保存违章证据。
进一步地，每隔预定时间或满足预设条件时将所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述第i帧图像及所述第i-1帧图像存储上传到远程服务器，并在所述上传完成后删除本地存储，能在系统空闲时完成违章证据的上传，保障了上传的成功率、并且上传成功后删除本地存储，进一步释放了本地存储空间。
图3是本申请一种实施例的实时车辆违章检测装置的结构示意图。如图3所示，所述实时车辆违章检测装置20包括视频接收模块21、视频缓存模块22、信息提取模块23、违章判断模块24、违章信息提取模块25、违章信息存储模块26及违章信息发送模块27。
视频接收模块21，用于获取实时视频图像；本实施例中，所述视频接收模块21可以接收车载摄像头在行车过程中拍摄的视频图像。
视频缓存模块22，用于将所述视频接收模块21获取的所述视频图像进行缓存；当所述视频图像超过所述视频缓存模块22的存储容量时可以删除最先存储的至少部分所述视频图像。
信息提取模块23，用于对所述视频接收模块21获取的所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数，所述信息提取模块23可以对每帧图像采用图像识别技术将车辆、车道线作为目标对象分别进行识别，通过识别获取每帧图像中的各车道及其车道类型、各车辆及其车辆属性，本实施例中，所述车辆属性包括颜色、车型、品牌、车牌中一个、两个或多个，所述车道类型包括实线道、导流线道、公交车道中的一个、两个或多个。并依据所述车辆属性生成各车辆的身份标识。
通过图像识别技术，所述信息提取模块23可以在每帧图像中分别识别出所述车辆、所述车道线，并将所述车辆、所述车道线从图像中分割出来，分别获取到所述车辆分割图像与所述车道分割图像。
具体的，将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对，获得第一相似性比对结果，所述第一相似性比对结果包括所述第i-1帧图像的每一车辆与所述第i帧图像的至少一辆车辆为相似车辆；将所述第i-1帧图像的每一车辆与所述第i帧图像中与其相对应的所述相似车辆中的一辆进行随机匹配或与所述第i帧图像中未匹配车辆进行剩余匹配；
依据所述第一相似性比对结果判断所述剩余匹配结果是否合理，若合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像，若不合理，则将所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的其他一辆所述相似车辆进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像。
所述信息提取模块23对所述车道分割匹配图像获取过程与上述所述车辆分割匹配图像的获取过程类似，即将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对，获得第二相似性比对结果，所述第二相似性比对结果包括所述第i-1帧图像的每一车道与所述第i帧图像的至少一条车道为相似车道；将所述第i-1帧图像的每一车道与所述第i帧图像中与其相对应的所述相似车道中的一条进行随机匹配或与所述第i帧图像中未匹配车道进行剩余匹配，
依据所述第二相似性比对结果判断所述剩余匹配结果是否合理；若合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像，若不合理，则将所述第i-1帧图像的车道与所述第i帧图像中与其相对应的其他一条所述相似车道进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像。
违章判断模块24，用于依据所述信息提取模块23提取的所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；当判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取，具体的，所述违章判断模块24对所述第i帧图像中是否有车辆违章进行判断可以依据所述车道分割匹配图像中的各车道的车道属性并结合所述车辆分割匹配图像中车辆匹配线判断是否有车辆违章，若所述车道分割匹配图像中具有预设车道属性的车道，且有车辆匹配线延伸至所述具有预设车道属性的车道与其他车道，则判断所述车辆匹配线对应的车辆是违章车辆。当检测到有车辆违章时，可以依据所述第i帧图像进行违章车辆的车辆信息提取。本实施例中，所述违章车辆的车辆信息可以为车辆的车牌号。
若所述违章判断模块24提取所述违章车辆的车辆信息成功，则对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像发送给所述违章信息存储模块26进行存储；违章信息发送模块27每隔预定时间或满足预设条件时从所述所述违章信息存储模块26中将所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像存储上传到远程服务器，并在所述上传完成后删除所述所述违章信息存储模块26存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像。
若所述违章判断模块24提取所述违章车辆信息失败，则将违章车辆的所述身份标识发送给所述违章信息提取模块25，所述违章信息提取模块25依据违章车辆的所述身份标识在所述视频接收模块21缓存的所述视频图像中获取对应的车辆信息，本实施例中，所述车辆信息可以是车牌号。同样，所述违章信息提取模块25在获取到所述车辆信息后，可以对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像发送给所述违章信息存储模块26进行存储；违章信息发送模块27每隔预定时间或满足预设条件时从所述所述违章信息存储模块26中将所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像存储上传到远程服务器，并在所述上传完成后删除所述所述违章信息存储模块26存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像。
可以理解的是，上述实时车辆违章检测装置的各模块实现各功能的具体方式可参阅上述实施例对应的具体步骤，故在此不作赘述。
请参阅图4，图4是本申请一种实施例的实时车辆违章检测设备的结构示意图。如图4所示，所述实时车辆违章检测设备30包括存储器32、处理器31及存储在所述存储器32上并可在所述处理器31上运行的计算机程序，所述处理器31执行所述计算机程序时实现所述实时车辆违章检测方法。
参阅图5，图5是本申请一种实施例的存储介质的结构示意图。如图5所示存储有计算机可读指令41的存储介质，该计算机可读指令41被一个或多个处理器执行时，使得一个或多个处理器执行所述实时车辆违章检测方法。 
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，该计算机程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，前述的存储介质可为磁碟、光盘、只读存储记忆体（Read-Only Memory，ROM）等非易失性存储介质，或随机存储记忆体（Random Access Memory，RAM）等易失性存储介质。
以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (20)

1、一种实时车辆违章检测方法，其中，所述实时车辆违章检测方法包括以下步骤：
获取实时视频图像；
将所述视频图像进行缓存；
对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数；
依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；
若判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取；
若所述车辆信息提取成功，则发送提取的所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报；
若所述车辆信息提取失败，则获取所述违章车辆的身份标识；及
依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息，发送所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
2、根据权利要求1所述实时车辆违章检测方法，其中，所述对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像包括：
识别每帧图像中的各车道及其车道类型、各车辆及其车辆属性，所述车辆属性包括颜色、车型、品牌、车牌中一个、两个或多个，所述车道类型包括实线道、导流线道、公交车道中的一个、两个或多个；
依据所述车辆属性生成各车辆的身份标识；
依据所述识别结果分割每帧图像获得车辆分割图像与车道分割图像；
对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像；及
对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像。
3、根据权利要求2所述实时车辆违章检测方法，其中，对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像，包括：
将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对，获得第一相似性比对结果，所述第一相似性比对结果包括所述第i-1帧图像的每一车辆与所述第i帧图像的至少一辆车辆为相似车辆；
将所述第i-1帧图像的每一车辆与所述第i帧图像中与其相对应的所述相似车辆中的一辆进行随机匹配或与所述第i帧图像中未匹配车辆进行剩余匹配；
依据所述第一相似性比对结果判断所述剩余匹配结果是否合理，
若合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像，
若不合理，则将所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的其他一辆所述相似车辆进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像。
4、根据权利要求2所述实时车辆违章检测方法，其中，对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像，包括：
将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对，获得第二相似性比对结果，所述第二相似性比对结果包括所述第i-1帧图像的每一车道与所述第i帧图像的至少一条车道为相似车道；
将所述第i-1帧图像的每一车道与所述第i帧图像中与其相对应的所述相似车道中的一条进行随机匹配或与所述第i帧图像中未匹配车道进行剩余匹配，
依据所述第二相似性比对结果判断所述剩余匹配结果是否合理；
若合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像，
若不合理，则将所述第i-1帧图像的车道与所述第i帧图像中与其相对应的其他一条所述相似车道进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像。
5、根据权利要求2所述实时车辆违章检测方法，其中，所述依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章，包括：
依据所述车道分割匹配图像中的各车道的车道属性并结合所述车辆分割匹配图像中车辆匹配线判断是否有车辆违章，若所述车道分割匹配图像中具有预设车道属性的车道，且有车辆匹配线延伸至所述具有预设车道属性的车道与其他车道，则判断所述车辆匹配线对应的车辆是违章车辆。
6、根据权利要求1所述实时车辆违章检测方法，其中，所述将所述视频图像进行缓存，还包括：
当所述视频图像超过存储容量时删除最先存储的至少部分所述视频图像；所述至少部分所述视频图像为一帧视频图像。
7、根据权利要求1所述实时车辆违章检测方法，其中，所述实时车辆违章检测方法还包括：
对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行存储；及
每隔预定时间或满足预设条件时将所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像存储上传到远程服务器，并在所述上传完成后删除所述存储的所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像。
8、一种实时车辆违章检测装置，其中，包括：
视频接收模块，用于获取实时视频图像；
视频缓存模块，用于将所述视频图像进行缓存；
信息提取模块，用于对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数；
违章判断模块，用于依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；当判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取；
违章信息提取模块，用于当所述违章判断模块提取所述违章车辆的车辆信息提取失败时，获取所述违章车辆的身份标识；及依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息；
违章信息存储模块，用于对所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行存储；
违章信息发送模块，用于发送所述违章上报涉及的车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
9、一种实时车辆违章检测设备，其中，所述实时车辆违章检测设备包括处理器、与所述处理器耦接的存储器，其中，
所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行以下步骤：
获取实时视频图像；
将所述视频图像进行缓存；
对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数；
依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；
若判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取；
若所述车辆信息提取成功，则发送提取的所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报；
若所述车辆信息提取失败，则获取所述违章车辆的身份标识；及
依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息，发送所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
10、根据权利要求9所述实时车辆违章检测设备，其中，所述对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像包括：
识别每帧图像中的各车道及其车道类型、各车辆及其车辆属性，所述车辆属性包括颜色、车型、品牌、车牌中一个、两个或多个，所述车道类型包括实线道、导流线道、公交车道中的一个、两个或多个；
依据所述车辆属性生成各车辆的身份标识；
依据所述识别结果分割每帧图像获得车辆分割图像与车道分割图像；
对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像；及
对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像。
11、根据权利要求10所述实时车辆违章检测设备，其中，对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像，包括：
将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对，获得第一相似性比对结果，所述第一相似性比对结果包括所述第i-1帧图像的每一车辆与所述第i帧图像的至少一辆车辆为相似车辆；
将所述第i-1帧图像的每一车辆与所述第i帧图像中与其相对应的所述相似车辆中的一辆进行随机匹配或与所述第i帧图像中未匹配车辆进行剩余匹配；
依据所述第一相似性比对结果判断所述剩余匹配结果是否合理，
若合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像，
若不合理，则将所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的其他一辆所述相似车辆进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像。
12、根据权利要求10所述实时车辆违章检测设备，其中，对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像，包括：
将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对，获得第二相似性比对结果，所述第二相似性比对结果包括所述第i-1帧图像的每一车道与所述第i帧图像的至少一条车道为相似车道；
将所述第i-1帧图像的每一车道与所述第i帧图像中与其相对应的所述相似车道中的一条进行随机匹配或与所述第i帧图像中未匹配车道进行剩余匹配，
依据所述第二相似性比对结果判断所述剩余匹配结果是否合理；
若合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像，
若不合理，则将所述第i-1帧图像的车道与所述第i帧图像中与其相对应的其他一条所述相似车道进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像。
13、根据权利要求10所述实时车辆违章检测设备，其中，所述依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章，包括：
依据所述车道分割匹配图像中的各车道的车道属性并结合所述车辆分割匹配图像中车辆匹配线判断是否有车辆违章，若所述车道分割匹配图像中具有预设车道属性的车道，且有车辆匹配线延伸至所述具有预设车道属性的车道与其他车道，则判断所述车辆匹配线对应的车辆是违章车辆。
14、根据权利要求9所述实时车辆违章检测设备，其中，所述将所述视频图像进行缓存，还包括：
当所述视频图像超过存储容量时删除最先存储的至少部分所述视频图像；所述至少部分所述视频图像为一帧视频图像。
15、一种存储有计算机可读指令的存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行以下步骤：
获取实时视频图像；
将所述视频图像进行缓存；
对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像，其中i为大于等于2的自然数；
依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章；
若判断有车辆违章，则依据所述第i帧图像进行违章车辆的车辆信息提取；
若所述车辆信息提取成功，则发送提取的所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报；
若所述车辆信息提取失败，则获取所述违章车辆的身份标识；及
依据所述身份标识在缓存的所述视频图像中获取对应的车辆信息，发送所述车辆信息及所述第i帧图像或所述车辆信息及所述第i帧图像及所述第i-1帧图像进行违章上报。
16、根据权利要求15所述存储有计算机可读指令的存储介质，其中，所述对所述视频图像中的第i帧图像及第i-1帧图像进行车辆、车道线识别，并对识别的各车辆生成身份标识，以及依据识别结果对所述第i帧图像及所述第i-1帧图像进行车辆、车道的分割与匹配，获得车辆分割匹配图像与车道分割匹配图像包括：
识别每帧图像中的各车道及其车道类型、各车辆及其车辆属性，所述车辆属性包括颜色、车型、品牌、车牌中一个、两个或多个，所述车道类型包括实线道、导流线道、公交车道中的一个、两个或多个；
依据所述车辆属性生成各车辆的身份标识；
依据所述识别结果分割每帧图像获得车辆分割图像与车道分割图像；
对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像；及
对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像。
17、根据权利要求16所述存储有计算机可读指令的存储介质，其中，对所述第i帧图像及所述第i-1帧图像的车辆分割图像进行车辆匹配获得所述车辆分割匹配图像，包括：
将所述第i-1帧图像的车辆分割图像中的各车辆与所述第i帧图像的车辆分割图像中的各车辆进行相似性比对，获得第一相似性比对结果，所述第一相似性比对结果包括所述第i-1帧图像的每一车辆与所述第i帧图像的至少一辆车辆为相似车辆；
将所述第i-1帧图像的每一车辆与所述第i帧图像中与其相对应的所述相似车辆中的一辆进行随机匹配或与所述第i帧图像中未匹配车辆进行剩余匹配；
依据所述第一相似性比对结果判断所述剩余匹配结果是否合理，
若合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像，
若不合理，则将所述第i-1帧图像的车辆与所述第i帧图像中与其相对应的其他一辆所述相似车辆进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车辆分割图像拼接在一起并将匹配的车辆加载匹配线，从而获得所述车辆分割匹配图像。
18、根据权利要求16所述存储有计算机可读指令的存储介质，其中，对所述第i帧图像及所述第i-1帧图像的车道分割图像进行车道匹配获得所述车道分割匹配图像，包括：
将所述第i-1帧图像的车道分割图像中的各车道与所述第i帧图像的车道分割图像中的各车道进行相似性比对，获得第二相似性比对结果，所述第二相似性比对结果包括所述第i-1帧图像的每一车道与所述第i帧图像的至少一条车道为相似车道；
将所述第i-1帧图像的每一车道与所述第i帧图像中与其相对应的所述相似车道中的一条进行随机匹配或与所述第i帧图像中未匹配车道进行剩余匹配，
依据所述第二相似性比对结果判断所述剩余匹配结果是否合理；
若合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像，
若不合理，则将所述第i-1帧图像的车道与所述第i帧图像中与其相对应的其他一条所述相似车道进行匹配，直到所述剩余匹配的结果判断为合理，则将所述第i帧图像及所述第i-1帧图像的车道分割图像拼接在一起并将匹配的车道加载匹配线，从而获得所述车道分割匹配图像。
19、根据权利要求16所述存储有计算机可读指令的存储介质，其中，所述依据所述车辆分割匹配图像与所述车道分割匹配图像判断所述第i帧图像中是否有车辆违章，包括：
依据所述车道分割匹配图像中的各车道的车道属性并结合所述车辆分割匹配图像中车辆匹配线判断是否有车辆违章，若所述车道分割匹配图像中具有预设车道属性的车道，且有车辆匹配线延伸至所述具有预设车道属性的车道与其他车道，则判断所述车辆匹配线对应的车辆是违章车辆。
20、根据权利要求15所述存储有计算机可读指令的存储介质，其中，所述将所述视频图像进行缓存，还包括：
当所述视频图像超过存储容量时删除最先存储的至少部分所述视频图像；所述至少部分所述视频图像为一帧视频图像。
PCT/CN2020/118438 2020-05-28 2020-09-28 实时车辆违章检测方法、装置、设备及存储介质 WO2021120776A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010470098.2A CN111666853B (zh) 2020-05-28 2020-05-28 实时车辆违章检测方法、装置、设备及存储介质
CN202010470098.2 2020-05-28

Publications (1)

Publication Number Publication Date
WO2021120776A1 true WO2021120776A1 (zh) 2021-06-24

Family

ID=72385108

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/118438 WO2021120776A1 (zh) 2020-05-28 2020-09-28 实时车辆违章检测方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN111666853B (zh)
WO (1) WO2021120776A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642412A (zh) * 2021-07-16 2021-11-12 盛视科技股份有限公司 占用公交车道的车辆检测方法、装置及设备
CN113989300A (zh) * 2021-10-29 2022-01-28 北京百度网讯科技有限公司 车道线分割的方法、装置、电子设备和存储介质
WO2022267263A1 (zh) * 2021-06-25 2022-12-29 北京市商汤科技开发有限公司 交通违章事件取证方法、装置、设备及系统

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111666853B (zh) * 2020-05-28 2023-06-02 平安科技（深圳）有限公司 实时车辆违章检测方法、装置、设备及存储介质
CN112257555A (zh) * 2020-10-20 2021-01-22 平安科技（深圳）有限公司 一种信息处理方法、装置、设备及存储介质
CN112348992B (zh) * 2020-11-12 2021-06-29 广东利通科技投资有限公司 基于车路协同系统的车载视频处理方法、装置及存储介质
CN112712703A (zh) * 2020-12-09 2021-04-27 上海眼控科技股份有限公司 车辆视频的处理方法、装置、计算机设备和存储介质
CN112712708A (zh) * 2020-12-28 2021-04-27 上海眼控科技股份有限公司 信息检测方法、装置、设备及存储介质
CN114387533A (zh) * 2022-01-07 2022-04-22 北京远度互联科技有限公司 道路违规识别的方法、装置、电子设备及存储介质
CN115206091B (zh) * 2022-06-07 2024-06-07 西安电子科技大学广州研究院 基于多摄像头和毫米波雷达的路况及事件监测系统及方法
CN115359665A (zh) * 2022-08-17 2022-11-18 北京博宏科元信息科技有限公司 基于射频视频一体机的多通道违章车辆记录方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3128496A1 (en) * 2014-12-30 2017-02-08 Tsinghua University Vehicle identification method and system
CN106652465A (zh) * 2016-11-15 2017-05-10 成都通甲优博科技有限责任公司 一种道路异常驾驶行为的识别方法及系统
CN106991820A (zh) * 2016-01-20 2017-07-28 中兴通讯股份有限公司 违规车辆处理方法及装置
US20190057244A1 (en) * 2017-08-18 2019-02-21 Autel Robotics Co., Ltd. Method for determining target through intelligent following of unmanned aerial vehicle, unmanned aerial vehicle and remote control
CN109740550A (zh) * 2019-01-08 2019-05-10 哈尔滨理工大学 一种基于单目视觉的车道线检测与跟踪方法
CN110610153A (zh) * 2019-09-10 2019-12-24 重庆工程职业技术学院 一种用于自动驾驶的车道识别方法及系统
CN111666853A (zh) * 2020-05-28 2020-09-15 平安科技（深圳）有限公司 实时车辆违章检测方法、装置、设备及存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105185114A (zh) * 2015-09-07 2015-12-23 北京信路威科技股份有限公司 一种车辆违章变道行为取证装置及方法
CN106652468B (zh) * 2016-12-09 2019-09-10 武汉极目智能技术有限公司 车辆道路前车违规检测和自车违规预警提醒装置及方法
CN107481526A (zh) * 2017-09-07 2017-12-15 公安部第三研究所 用于行车变道检测记录及违章变道举报控制的系统及方法
JP7218535B2 (ja) * 2018-10-12 2023-02-07 トヨタ自動車株式会社 交通違反車両識別システム及びサーバ
CN110706261A (zh) * 2019-10-22 2020-01-17 上海眼控科技股份有限公司 车辆违章检测方法、装置、计算机设备和存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3128496A1 (en) * 2014-12-30 2017-02-08 Tsinghua University Vehicle identification method and system
CN106991820A (zh) * 2016-01-20 2017-07-28 中兴通讯股份有限公司 违规车辆处理方法及装置
CN106652465A (zh) * 2016-11-15 2017-05-10 成都通甲优博科技有限责任公司 一种道路异常驾驶行为的识别方法及系统
US20190057244A1 (en) * 2017-08-18 2019-02-21 Autel Robotics Co., Ltd. Method for determining target through intelligent following of unmanned aerial vehicle, unmanned aerial vehicle and remote control
CN109740550A (zh) * 2019-01-08 2019-05-10 哈尔滨理工大学 一种基于单目视觉的车道线检测与跟踪方法
CN110610153A (zh) * 2019-09-10 2019-12-24 重庆工程职业技术学院 一种用于自动驾驶的车道识别方法及系统
CN111666853A (zh) * 2020-05-28 2020-09-15 平安科技（深圳）有限公司 实时车辆违章检测方法、装置、设备及存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022267263A1 (zh) * 2021-06-25 2022-12-29 北京市商汤科技开发有限公司 交通违章事件取证方法、装置、设备及系统
CN113642412A (zh) * 2021-07-16 2021-11-12 盛视科技股份有限公司 占用公交车道的车辆检测方法、装置及设备
CN113642412B (zh) * 2021-07-16 2023-12-26 盛视科技股份有限公司 占用公交车道的车辆检测方法、装置及设备
CN113989300A (zh) * 2021-10-29 2022-01-28 北京百度网讯科技有限公司 车道线分割的方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN111666853A (zh) 2020-09-15
CN111666853B (zh) 2023-06-02

Similar Documents

Publication Publication Date Title
WO2021120776A1 (zh) 实时车辆违章检测方法、装置、设备及存储介质
JP7218535B2 (ja) 交通違反車両識別システム及びサーバ
WO2020248386A1 (zh) 视频分析方法、装置、计算机设备及存储介质
WO2019105342A1 (zh) 车辆套牌检测方法、装置、可读存储介质及电子设备
CN104332052B (zh) 一种行人闯红灯自动抓拍的识别方法
CN108806272B (zh) 一种提醒多辆机动车车主违法停车行为的方法及装置
CN109003338A (zh) 一种路侧停车自动计时收费方法及装置
US20170032514A1 (en) Abandoned object detection apparatus and method and system
CN110580808B (zh) 一种信息处理方法、装置、电子设备及智能交通系统
WO2021184570A1 (zh) 电动自行车驾驶行为的识别方法、装置和计算机设备
CN108932851B (zh) 一种记录机动车违法停车行为的方法及装置
CN109377694B (zh) 社区车辆的监控方法及系统
CN108932852B (zh) 一种记录机动车违法占用应急车道行为的方法及装置
CN106097722A (zh) 利用视频进行路侧车位自动化监管的系统和方法
CN110276258A (zh) 一种识别车辆外观的方法及系统
CN111292530A (zh) 处理违章图片的方法、装置、服务器和存储介质
WO2014063524A1 (zh) 一种检测摄像机是否被干扰的方法、装置及视频监控系统
CN108932850B (zh) 一种记录机动车低速行驶违法行为的方法及装置
CN113990101B (zh) 一种禁停区域车辆检测方法及其系统和处理装置
CN115909313A (zh) 一种基于深度学习的违停车牌识别方法和装置
CN110660225A (zh) 闯红灯行为检测方法、装置和设备
CN109003457B (zh) 一种记录多台机动车违法占用应急车道行为的方法及装置
CN112084905B (zh) 交通灯状态识别方法、系统、设备及存储介质
CN112232441B (zh) 违章停车判断方法、系统、计算机设备及存储介质
CN106530727A (zh) 交通违规信息的处理系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20901088

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20901088

Country of ref document: EP

Kind code of ref document: A1