WO2022227548A1 - Spill-out event detection method and apparatus, electronic device, storage medium, and computer program product - Google Patents

Spill-out event detection method and apparatus, electronic device, storage medium, and computer program product Download PDF

Info

Publication number
WO2022227548A1
WO2022227548A1 PCT/CN2021/133487 CN2021133487W WO2022227548A1 WO 2022227548 A1 WO2022227548 A1 WO 2022227548A1 CN 2021133487 W CN2021133487 W CN 2021133487W WO 2022227548 A1 WO2022227548 A1 WO 2022227548A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
frame
tested
detection
throwing
Prior art date
Application number
PCT/CN2021/133487
Other languages
French (fr)
Chinese (zh)
Inventor
王意如
甘伟豪
孙献峰
李七星
Original Assignee
北京市商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京市商汤科技开发有限公司 filed Critical 北京市商汤科技开发有限公司
Publication of WO2022227548A1 publication Critical patent/WO2022227548A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular, to a method, device, electronic device, storage medium and computer program product for detecting a throwing event.
  • the detection of motor vehicle foreign object throwing events is an analysis event with high usage requirements.
  • the detection method for the incident of vehicle foreign object throwing is mainly based on the target detection algorithm, and the throwing object is identified directly in the video image.
  • the detection of spillage events based on the target detection algorithm can only identify the spilled objects in the image, and the detected information is single.
  • Embodiments of the present disclosure are expected to provide a method, apparatus, electronic device, storage medium, and computer program product for detecting a spillage event.
  • Embodiments of the present disclosure provide a method for detecting a throwing event, the method comprising:
  • vehicle detection model uses the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested;
  • throwing event detection is performed based on a corresponding set of stitched images in the at least one set of stitched images, to obtain at least one throwing event detection result.
  • the obtaining the reference frame corresponding to the current frame includes:
  • vehicle detection is performed on each frame of the at least one frame of image, and the number of vehicles detected in each frame of image is counted;
  • a frame image with the least number of vehicles detected in the at least one frame image is determined as the reference frame.
  • the method before using the throwing event detection model to detect the throwing event of each vehicle to be tested in the at least one vehicle to be tested based on the at least one set of stitched images, the method further includes:
  • the throwing event detection model is used to perform throwing event detection for each vehicle to be tested in the at least one vehicle to be tested based on the at least one set of stitched images.
  • the use of the reference frame to judge the suspected throwing sample of the current frame to obtain a sample judgment result including:
  • Each pixel in the current frame and the pixel at the position corresponding to each pixel in the reference frame are compared on different color channels to obtain the absolute value, and the obtained values of the different color channels are compared.
  • the absolute difference is averaged to obtain the first difference matrix
  • the sample judgment result is a suspected throwing sample
  • the sample judgment result is a non-suspected throwing sample.
  • the method before the sample judgment is performed on the current frame by using the reference frame, and the sample judgment result is obtained, the method further includes;
  • the movement judgment of the shooting lens is performed to obtain a movement judgment result
  • the said reference frame is used to judge the suspected throwing sample of the current frame, and the sample judgment result is obtained, including:
  • the sample judgment result of the sample is obtained by using the reference frame to judge the suspected throwing sample of the current frame.
  • the movement judgment of the shooting lens is performed based on the current frame and the reference frame to obtain a movement judgment result, including:
  • the vehicle included in the reference frame and the element corresponding to the at least one vehicle to be tested in the current frame are replaced by zero, and the absolute value of the replaced difference matrix is obtained to obtain Processed difference matrix;
  • the background difference value is not greater than the preset difference threshold, it is determined that the movement determination result is no movement.
  • the images corresponding to the same position in the current frame and the reference frame are spliced to obtain the corresponding image of the at least one vehicle to be tested.
  • at least one set of stitched images including:
  • a set of detection candidate frames is generated around each vehicle to be tested of the at least one vehicle to be tested, to obtain at least one set of detection candidate frames;
  • the at least one set of detection candidate frames and the candidate frames corresponding to the positions of the at least one set of matching candidate frames are spliced to obtain the at least one set of spliced images.
  • a set of detection candidate frames is generated around each vehicle to be tested of the at least one vehicle to be tested, to obtain at least one set of detection candidate frames, including:
  • a first candidate frame is generated around the first vehicle; the first vehicle is any vehicle to be tested among the at least one vehicle to be tested;
  • a candidate frame that satisfies a preset size condition is selected from the reduced candidate frame, and the candidate frame that is not reduced in the first candidate frame forms a group of detection candidate frames corresponding to the first vehicle.
  • the reducing at least a part of the candidate frames in the first candidate frame includes:
  • the candidate frame located below the first vehicle is reduced according to a preset reduction scale.
  • the reducing at least a part of the candidate frames in the first candidate frame includes:
  • the method before the vehicle detection is performed on the current frame by using the vehicle detection model, the method further includes:
  • the method further includes:
  • the embodiment of the present disclosure provides a throwing event detection device, including:
  • a detection module configured to perform vehicle detection on the current frame by using a vehicle detection model, and determine each detected vehicle as a vehicle to be tested, to obtain at least one vehicle to be tested;
  • a splicing module configured to obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and an image corresponding to the same position in the reference frame, obtaining at least one group of stitched images corresponding to the at least one vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
  • the detection module is further configured to use a throwing event detection model to detect a throwing event based on a corresponding set of stitched images in the at least one set of stitched images for each vehicle to be tested in the at least one vehicle to be tested, and obtain: At least one spill event detection result.
  • the splicing module is specifically configured to acquire at least one frame of image collected within a preset time period before the current frame is collected; using the vehicle detection model, for each image in the at least one frame of image Vehicle detection is performed on the frame images respectively, and the number of vehicles detected in each frame image is counted; the frame image with the least number of vehicles detected in the at least one frame image is determined as the reference frame.
  • the detection module is further configured to use the reference frame to perform a sample judgment on the current frame suspected to be thrown, to obtain a sample judgment result;
  • the detection module is specifically configured to, when the sample judgment result is a suspected throwing sample, use the throwing event detection model, based on the at least one set of stitched images, to detect each of the at least one vehicle to be tested. Test vehicle for spill event detection.
  • the detection module is specifically configured to make a difference between each pixel in the current frame and a pixel at a position corresponding to each pixel in the reference frame on different color channels. Then take the absolute value, and take the mean value of the obtained absolute difference values of different color channels to obtain a first difference value matrix; perform low-pass filtering on the first difference value matrix to obtain a second difference value matrix; The second difference matrix takes the mean value to obtain the target evaluation mean value; in the case that the target evaluation mean value is greater than the preset mean value threshold, it is determined that the sample judgment result is a suspected throwing sample; when the target evaluation mean value is not greater than the predetermined mean value threshold In the case of setting the mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
  • the detection module is further configured to judge the movement of the shooting lens based on the current frame and the reference frame, and obtain a movement judgment result
  • the detection module is specifically configured to use the reference frame to judge a suspected throwing sample of the current frame when the movement judgment result is no movement, to obtain the sample judgment result.
  • the detection module is specifically configured to use the vehicle detection model to perform vehicle detection on the reference frame to obtain vehicles included in the reference frame; and compare each pixel in the current frame with the reference frame.
  • the pixel points in the reference frame corresponding to the positions of each pixel point are respectively different in different color channels, and the difference values of the obtained different color channels are averaged to obtain a difference matrix to be processed;
  • the difference matrix the element corresponding to the vehicle included in the reference frame and the at least one vehicle to be tested in the current frame is replaced by zero, and the absolute value of the replaced difference matrix is taken to obtain the processed difference matrix.
  • the splicing module is specifically configured to, in the current frame, generate a set of detection candidate frames around each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one set of detection candidate frames;
  • the reference frame at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames are selected respectively;
  • the corresponding candidate frames are spliced to obtain the at least one set of spliced images.
  • the splicing module is specifically configured to generate a first candidate frame around a first vehicle in the current frame; the first vehicle is any vehicle to be tested among the at least one vehicle to be tested; Reduce at least a part of the candidate frames in the first candidate frame; select candidate frames that meet the preset size condition from the reduced candidate frames, and form the first candidate frame with the unreduced candidate frames in the first candidate frame.
  • the splicing module is specifically configured to reduce the first candidate frame and the candidate frame below the first vehicle according to a preset reduction scale.
  • the splicing module is specifically configured to reduce the first candidate frame to a candidate frame that overlaps with a second candidate frame generated around a second vehicle; wherein the second vehicle is the at least one A vehicle to be tested that is different from the first vehicle among the vehicles to be tested.
  • the training module is configured to obtain vehicle detection samples and a preset detection model; perform vehicle detection training on the preset detection model by using the vehicle detection samples to obtain the vehicle detection model;
  • Embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a communication bus;
  • the communication bus configured to implement a communication connection between the processor and the memory
  • the processor is configured to execute one or more programs stored in the memory, so as to implement the above method for detecting a throwing event.
  • Embodiments of the present disclosure provide a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the above-mentioned throwing Event detection method.
  • An embodiment of the present disclosure provides a computer program product, the computer program product includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the above method for detecting a spillage event.
  • Embodiments of the present disclosure provide a throwing event detection method, device, electronic device, storage medium and computer program product.
  • the method includes: using a vehicle detection model to perform vehicle detection on a current frame, and determining each detected vehicle as a Vehicles to be tested, at least one vehicle to be tested is obtained; a reference frame corresponding to the current frame is obtained, and for each vehicle to be tested in at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are spliced to obtain at least one vehicle to be tested.
  • At least one group of stitched images corresponding to a vehicle to be tested the same position is the position around the vehicle to be tested in the current frame and the reference frame; using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, based on at least one
  • a set of spliced images corresponding to the set of spliced images is subjected to throwing event detection, and at least one throwing event detection result is obtained.
  • the throwing event detection method of the present disclosure uses a vehicle detection model to first perform vehicle detection from an image, and then uses the throwing event detection model to detect the throwing event based on the stitched images corresponding to the vehicle, which not only associates the throwing event with a specific vehicle, but also improves the Improved spill event detection performance.
  • FIG. 1 is a schematic flowchart of a method for detecting a throwing event according to an embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of an exemplary preset detection model provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of an exemplary first candidate frame provided by an embodiment of the present disclosure
  • FIG. 4 provides an exemplary mosaic image according to an embodiment of the present disclosure
  • FIG. 5 is a schematic structural diagram of a throwing event detection device according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • Embodiments of the present disclosure provide a method for detecting a spillage event, the execution body of which may be a device for detecting a spillage event.
  • the method for detecting a spillage event may be executed by a terminal device or a server or other electronic device, where the terminal device may be a user equipment ( User Equipment, UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (Personal Digital Assistant, PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the throwing event detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • FIG. 1 is a schematic flowchart of a method for detecting a throwing event according to an embodiment of the present disclosure. As shown in Figure 1, the method for detecting a spillage event mainly includes the following steps:
  • S101 Use a vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested.
  • the throwing event detection device when the throwing event detection device obtains the current frame, it can use the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested, and obtain at least one vehicle to be tested.
  • the throwing event detection apparatus may include a photographing lens for obtaining a current frame, which is a frame image currently required for throwing event detection, which may be currently collected.
  • the shooting lens and the throwing event detection device are two independent devices, and the shooting lens transmits the collected current frame to the throwing event detection model.
  • the specific source of the current frame is not limited in this embodiment of the present disclosure.
  • a vehicle detection model is stored in the throwing event detection device, which can perform vehicle detection on the input image and identify the vehicle in the input image.
  • the specific vehicle detection model is not limited in the embodiment of the present disclosure.
  • the throwing event detection device inputs the current frame into the vehicle detection module, and the vehicle detection model can locate a vehicle detection frame for each vehicle from the current frame, and each vehicle detection frame contains an image of a vehicle.
  • the throwing event detection device may further perform the following steps: acquiring a vehicle detection sample and a preset detection model; using the vehicle detection sample Perform vehicle detection training on the preset detection model to obtain a vehicle detection model.
  • the throwing event detection device obtains vehicle detection samples for training a preset detection model, thereby obtaining a vehicle detection model with higher accuracy and ensuring the accuracy of vehicle detection.
  • the vehicle detection model may be obtained after training the preset detection model by the vehicle detection sample by the throwing event detection device, and the specific vehicle detection sample and the preset detection model can be based on actual needs.
  • the settings and selections are not limited in the embodiments of the present disclosure.
  • FIG. 2 is a schematic structural diagram of an exemplary preset detection model provided by an embodiment of the present disclosure.
  • the preset detection model may include convolutional layers, pooling layers, and fully connected layers.
  • the throwing event detection device can input the vehicle detection samples into the preset detection model, and process them in sequence through different processing layers, so as to obtain the final vehicle detection result and compare the preset standard detection result, thereby adjusting the relevant parameters in the preset detection model , and finally get the vehicle detection model.
  • the throwing event detection device inputs the current frame into the vehicle detection model, so that each vehicle detected from the current frame is output from the vehicle detection model, that is, the vehicle to be tested.
  • the specific number of vehicles to be tested is not limited in the embodiment of the present disclosure.
  • S102 Acquire a reference frame corresponding to the current frame, and for each vehicle to be tested in at least one vehicle to be tested, splicing images corresponding to the same position of the current frame and the reference frame to obtain at least one set of splices corresponding to at least one vehicle to be tested Image; the same position is the position around the vehicle under test in the current frame and the reference frame.
  • the throwing event detection device can obtain the reference frame corresponding to the current frame when the current frame is obtained, so as to compare the current frame and the reference frame for each vehicle to be tested in the at least one vehicle to be tested Images corresponding to the same position are stitched to obtain at least one set of stitched images corresponding to at least one vehicle to be tested.
  • the throwing event detection device obtains a reference frame corresponding to the current frame, including: obtaining at least one frame of images collected within a preset time period between the current frames; using a vehicle detection model to detect Vehicle detection is performed for each frame of image in the at least one frame of image, and the number of vehicles detected in each frame of image is counted; the frame image with the least number of vehicles detected in the at least one frame of image is determined as a reference frame.
  • a preset time period is stored in the throwing event detection device, and the throwing event detection device may take the moment when the current frame is collected as a reference, and obtain the data collected within the preset duration between the moments. at least one frame of image.
  • the specific preset duration may be set according to actual requirements and application scenarios, which is not limited in the embodiment of the present disclosure.
  • the throwing event detection device may input each frame of image in at least one frame of image into the vehicle detection model, respectively, to perform vehicle detection, so as to count the number of vehicles included in each frame of image, and further
  • the frame image with the least number of vehicles is used as the reference frame corresponding to the current frame. This is because using the frame image with the least number of vehicles as the reference frame can minimize the interference of subsequent detection and avoid a large number of vehicles. occlusion, resulting in problems such as inability to correctly detect throwing events.
  • the throwing event detection apparatus may obtain the reference frame corresponding to the current frame in the above-mentioned manner, and may also determine the reference frame in other manners, for example, determining the image of the previous frame of the current frame as the reference frame frame.
  • the specific manner of acquiring the reference frame corresponding to the current frame may be set according to actual requirements and application scenarios, which is not limited in the embodiment of the present disclosure.
  • the throwing event detection device splices images corresponding to the same position in the current frame and the reference frame for each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one vehicle to be tested corresponding to at least one set of images, including: in the current frame, generating a set of detection candidate frames around each vehicle to be tested in at least one vehicle to be tested, to obtain at least one set of detection candidate frames; At least one set of matching candidate frames corresponding to one set of detection candidate frames; at least one set of detection candidate frames and at least one set of matching candidate frames are spliced with corresponding candidate frames to obtain at least one set of stitched images.
  • the throwing event detection device generates a set of detection candidate frames around each vehicle to be tested in at least one vehicle to be tested in the current frame, and obtains at least one set of candidate detection frames, including : In the current frame, a first candidate frame is generated around the first vehicle; the first vehicle is any vehicle to be tested in at least one vehicle to be tested; at least a part of the candidate frames in the first candidate frame is reduced; A candidate frame that satisfies the preset size condition is selected from the frame, and the candidate frame that is not reduced in the first candidate frame forms a set of detection candidate frames corresponding to the first vehicle.
  • the throwing event detection device generates a first candidate frame around the first vehicle in the current frame, which is actually generating a first candidate frame around the vehicle detection frame including the first vehicle .
  • FIG. 3 is a schematic diagram of an exemplary first candidate frame provided by an embodiment of the present disclosure.
  • the device for detecting a throwing event can use a vehicle detection model to detect the first vehicle from the current frame, so as to locate a vehicle detection frame including the first vehicle, and the vehicle detects the first vehicle.
  • the frame is a rectangular frame
  • the throwing event detection device specifically generates 12 first candidate frames around the vehicle detection frame
  • each first candidate frame is also a rectangular frame, wherein the first candidate frames located on both sides of the vehicle detection frame , the width is half the width of the vehicle detection frame, the first candidate frame located above and below the vehicle detection frame, and the length is half the length of the vehicle detection frame.
  • the specific number and size of the first candidate frame may be set according to actual requirements, which is not limited in the embodiment of the present disclosure.
  • the throwing event detection device since the detection frequency is relatively high in the actual scene of detection of the throwing event, and the farther from the vehicle to be tested, even if the throwing event is detected, it is different from the vehicle to be tested.
  • the correlation is also very small, that is, it is largely not the throwing event corresponding to the vehicle to be tested. Therefore, the throwing event detection device only generates a certain number of candidates according to the above size around the vehicle detection frame containing the first vehicle. In this way, in the subsequent detection of the throwing event, not only the detection efficiency is high, but also false detection can be avoided.
  • the throwing event detection device reduces at least a part of the candidate frames in the first candidate frame, including: reducing the candidate frames located under the first vehicle in the first candidate frame according to a preset scale down.
  • the throwing event detection device reduces the first candidate frame, the candidate frame located under the first vehicle, according to the preset reduction scale, which may be the first candidate frame,
  • the candidate frame below the vehicle detection frame containing the first vehicle is reduced in length and width by 10%, and its shape remains unchanged.
  • the specific reduction degree may be set according to actual requirements and application scenarios, which is not limited in the embodiment of the present disclosure.
  • the bottom of the vehicle detection frame including the first vehicle has a certain distance from the bottom of the body of the first vehicle, and the throwing event is performed at a position far away from the vehicle detection frame.
  • the detection actually has no correlation with the first vehicle. Therefore, the throwing event detection device can appropriately reduce the size of the candidate frame below the vehicle detection frame, thereby avoiding false detection.
  • the throwing event detection device reduces at least a part of the candidate frames in the first candidate frame, including: reducing the first candidate frame, and the second candidate frame overlapped with the second candidate frame generated around the second vehicle Candidate frame; wherein, the second vehicle is a vehicle to be tested that is different from the first vehicle in at least one vehicle to be tested.
  • the throwing event detection device can generate corresponding candidate frames around them.
  • the candidate frames around the detection frame may overlap, and when a throwing event is detected in the overlapping area, it is actually difficult to accurately distinguish the corresponding vehicle to be tested. Therefore, the throwing event detection device reduces the first candidate frame and generates it with the surrounding of other vehicles to be tested.
  • the candidate frame overlaps the candidate frame, so that subsequent detection can avoid false detection and obtain the optimal detection result.
  • the throwing event detection apparatus reduces at least a part of the candidate frames in the first candidate frame, and may further include: The candidate frame below the vehicle is reduced according to the preset reduction scale, and the first candidate frame is reduced to a candidate frame that overlaps with the second candidate frame generated around the second vehicle; wherein the second vehicle is at least one vehicle to be tested A vehicle to be tested that is different from the first vehicle.
  • the throwing event detection device can use any one of the above two methods to reduce the candidate frame to reduce the candidate frame.
  • the throwing event detection device can also use the above two methods at the same time.
  • the candidate frame reduction is performed by reducing the candidate frame, which is not limited in the embodiment of the present disclosure.
  • the throwing event detection device selects each candidate frame whose size satisfies the preset size condition from the reduced candidate frame, and the candidate frame that is not reduced in the first candidate frame.
  • a set of detection candidate boxes corresponding to the first vehicle is formed.
  • the specific preset size conditions may include preset length, preset width, preset aspect ratio, etc., which are not limited in the embodiment of the present disclosure.
  • the throwing event detection device generates a set of detection candidate frames for each vehicle to be tested in at least one vehicle to be tested, and for each candidate frame of each set of detection candidate frames , the candidate frame corresponding to the position can be obtained from the reference frame, so as to form a corresponding set of matching candidate frames.
  • the device for detection of the throwing event may correspond to each vehicle to be tested.
  • Each candidate frame in a set of detection candidate frames is spliced with the candidate frame corresponding to the position in the corresponding set of matching candidate frames, and a spliced image corresponding to the position is obtained.
  • FIG. 4 is an exemplary mosaic image provided by an embodiment of the present disclosure. As shown in Figure 4, the left and right sides of the spliced image are candidate frames at the same position in the current frame and the reference frame respectively, and splicing can be achieved between the two to obtain a spliced image.
  • the throwing event detection device can use the throwing event detection model to detect at least one vehicle to be tested based on the at least one set of stitched images when at least one set of stitched images corresponding to at least one vehicle to be tested is obtained. Detecting a spilling event is performed, and at least one spilling event detection result is obtained.
  • the throwing event detection device utilizes the throwing event detection model, for each vehicle to be tested in at least one vehicle to be tested, based on a corresponding set of stitched images in at least one set of stitched images.
  • the following steps may also be performed: obtaining a vehicle throwing sample and a preset time sequence difference neural network; using the vehicle throwing sample to train the vehicle throwing event detection on the preset time sequence difference neural network to obtain a throwing event detection model.
  • the throwing event detection device obtains vehicle throwing samples, which are used for training the preset time sequence difference neural network, so that a throwing event detection model with higher accuracy can be obtained to ensure the throwing event. detection accuracy.
  • the sprinkling event detection model is essentially a trained time-series difference neural network, which can obtain image information at the same location and at different times from the stitched images, so that the sprinkling event detection can be performed accurately, which is simple and efficient.
  • the throwing event detection model may be obtained by the throwing event detection device using the vehicle throwing samples to train the preset time sequence difference neural network.
  • the specific vehicle throwing samples and the preset timing difference are obtained.
  • the structure of the value neural network can be set and selected according to actual requirements, which is not limited in the embodiment of the present disclosure.
  • the throwing event detection device can input the stitched images into the throwing event detection model, and for each stitched image group in the set of stitched images Compare the difference between the two parts of the image stitching, output the difference score, and then determine whether the vehicle under test has a throwing event according to the difference score. For example, when the difference score exceeds a preset difference threshold, it is determined that the vehicle under test has a throwing event. , when the difference score is less than the preset difference threshold, it is determined that the vehicle to be tested does not have a throwing event.
  • the throwing event detection device uses the throwing event detection model to throw each vehicle under test in at least one vehicle under test based on at least one set of stitched images.
  • the following steps may also be performed: use the reference frame to judge the suspected throwing sample of the current frame, and obtain the sample judgment result.
  • the throwing event detection device uses a throwing event detection model to detect throwing events for each vehicle to be tested in at least one vehicle to be tested based on at least one set of stitched images, which may include: when the sample judgment result is a suspected throwing sample, using throwing An event detection model for detecting a throwing event for each vehicle to be tested in the at least one vehicle to be tested based on at least one set of stitched images.
  • the throwing event detection device uses the reference frame to judge the suspected throwing sample of the current frame, and obtains the sample judgment result, including: comparing each pixel in the current frame with each pixel in the reference frame and each pixel The pixel points at the corresponding position of the point, take the absolute value after making the difference on different color channels, and take the average value of the absolute difference value of the obtained different color channels to obtain the first difference value matrix; perform low-pass on the first difference value matrix.
  • the throwing event detection device can take the absolute value of each pixel in the current frame and the pixel at the corresponding position in the reference frame after making a difference on the RGB channel, and calculate the difference. Further take the mean value in the RGB channel to obtain the first difference matrix. Afterwards, the throwing event detection device performs low-pass filtering on the first difference matrix and takes an average value to obtain the target evaluation average value. If the target evaluation mean is greater than the preset mean threshold, that is, the sample judgment result is a suspected throwing sample, which means that the overall image information of the current frame and the reference frame is quite different. Therefore, the vehicle under test included in the current frame is likely to have a throwing event, that is, it can be Step S103 is performed.
  • the target evaluation mean is not greater than the preset mean threshold, that is, the sample judgment result is a non-suspected throwing sample, it means that the overall image information of the current frame and the reference frame is basically the same. Therefore, the vehicle under test included in the current frame has a high probability that there is no throwing event.
  • the subsequent throwing event detection device may not perform throwing event detection on the vehicle to be tested, that is, step S103 may not be performed to avoid unnecessary detection, thereby improving detection efficiency and reducing power consumption of the throwing event detection device.
  • the throwing event detection device uses the reference frame to detect the suspected throwing sample of the current frame, and before obtaining the sample judgment result, the following steps may also be performed: taking a shot based on the current frame and the reference frame. The movement judgment is obtained, and the movement judgment result is obtained. The throwing event detection device uses the reference frame to judge the suspected throwing sample of the current frame, and obtains the sample judgment result, including: when the movement judgment result is no movement, using the reference frame to judge the suspected throwing sample of the current frame, and obtain the sample judgment result .
  • the throwing event detection device determines the movement of the shooting lens based on the current frame and the reference frame, and obtains the movement determination result.
  • Vehicle make a difference between each pixel point in the current frame and the pixel point corresponding to each pixel point in the reference frame on different color channels, and take the average of the obtained difference values of different color channels to obtain the pending processing Difference matrix; replace the elements corresponding to the vehicles included in the reference frame and at least one vehicle to be tested in the current frame in the difference matrix to be processed to zero, and take the absolute value of the replaced difference matrix to obtain the processed difference matrix; calculate the average value of the processed difference matrix to obtain the background difference value; in the case that the background difference value is greater than the preset difference threshold, it is determined that the movement judgment result is movement; in the case that the background difference value is not greater than the preset difference threshold , and determine that the movement judgment result is no movement.
  • the throwing event detection device may make a difference between each pixel in the current frame and the pixel at the corresponding position in the reference frame on the RGB channel, and further detect the difference between the RGB channel. Take the mean to get the difference matrix to be processed. Since the judgment of the movement of the shooting lens is based on the difference of the image background, the elements corresponding to the two frames of images in the difference matrix to be processed are replaced by zero, and the absolute value is further taken to obtain the processed difference matrix, and finally the processed difference matrix is processed. Calculate the mean of the value matrix to obtain the background difference value.
  • the background difference value is greater than the preset difference threshold, that is, the result of the movement judgment is movement, it means that the shooting lens is not in the same position when collecting the current frame and the reference frame, and the shooting lens moves during the process of collecting the current frame and the reference frame, and the reference frame It is not an image at the same position at a different time from the current frame, so the reference frame cannot be used for subsequent throwing event detection.
  • the background difference value is not greater than the preset difference threshold, that is, the movement judgment result is no movement, it means that the shooting lens is in the same position when collecting the current frame and the reference frame, and the shooting lens does not move during the process of collecting the current frame and the reference frame.
  • the frame and the current frame are images of the same position at different times, so the reference frame can be used to detect subsequent throwing events, thereby improving the detection efficiency.
  • a preset difference threshold is stored in the throwing event detection device, which is used to measure the background difference degree between the reference frame and the current frame.
  • the specific preset difference threshold may be set according to actual requirements and application scenarios, which is not limited in this embodiment of the present disclosure.
  • the throwing event detection device may use the above method to determine the movement of the shooting lens, or may also use other methods to determine, for example, correspondingly select four sides from the current frame and the reference frame. Differential comparison of image areas at corner positions.
  • the specific manner of judging the movement of the shooting lens is not limited in the embodiment of the present disclosure.
  • An embodiment of the present disclosure provides a method for detecting a throwing event, including: using a vehicle detection model to perform vehicle detection on a current frame, and determining each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested;
  • the reference frame corresponding to the frame, and for each vehicle to be tested in the at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are stitched to obtain at least one set of stitched images corresponding to at least one vehicle to be tested; the same The position is the position around the vehicle to be tested in the current frame and the reference frame; using the throwing event detection model, for each vehicle to be tested in at least one vehicle to be tested, the throwing event is performed based on a corresponding set of stitched images in at least one set of stitched images Detect, and obtain at least one throwing event detection result.
  • the throwing event detection method of the present disclosure uses a vehicle detection model to first perform vehicle detection from an image, and then uses the throwing event detection model to detect the throwing event based on the stitched images corresponding to the vehicle, which not only associates the throwing event with a specific vehicle, but also improves the Improved spill event detection performance.
  • FIG. 5 is a schematic structural diagram of a throwing event detection device according to an embodiment of the present disclosure. As shown in Figure 5, the throwing event detection device includes:
  • the detection module 501 is configured to use the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested, to obtain at least one vehicle to be tested;
  • the splicing module 502 is configured to obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and the image corresponding to the same position in the reference frame to obtain at least one group of stitched images corresponding to the at least one vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
  • the detection module 501 is further configured to use a throwing event detection model to detect a throwing event based on a corresponding set of stitched images in the at least one set of stitched images for each vehicle to be tested in the at least one vehicle to be tested, Obtain at least one spill event detection result.
  • the splicing module 502 is specifically configured to acquire at least one frame of image collected within a preset time period before the current frame is collected; Vehicle detection is performed on each frame of image in the image, and the number of vehicles detected in each frame of image is counted; the frame image with the least number of vehicles detected in the at least one frame of image is determined as the reference frame .
  • the detection module 501 is further configured to use the reference frame to perform a sample judgment on the current frame suspected to be thrown, to obtain a sample judgment result;
  • the detection module 501 is specifically configured to, when the sample judgment result is a suspected throwing sample, use the throwing event detection model to detect each of the at least one vehicle to be tested based on the at least one set of stitched images.
  • the vehicle to be tested is tested for spillage events.
  • the detection module 501 is specifically configured to compare each pixel in the current frame and a pixel at a position corresponding to each pixel in the reference frame in different color channels Take the absolute value after making the difference respectively, and take the mean value of the absolute difference values of the obtained different color channels to obtain the first difference value matrix; perform low-pass filtering processing on the first difference value matrix to obtain the second difference value matrix Take the mean value of the second difference matrix to obtain the target evaluation mean value; in the case that the target evaluation mean value is greater than the preset mean value threshold, it is determined that the sample judgment result is a suspected throwing sample; when the target evaluation mean value is not When the value is greater than the preset mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
  • the detection module 501 is further configured to determine the movement of the shooting lens based on the current frame and the reference frame, and obtain a movement determination result;
  • the detection module 501 is specifically configured to use the reference frame to perform a sample judgment on a suspected throwing sample of the current frame when the movement judgment result is no movement, to obtain the sample judgment result.
  • the detection module 501 is specifically configured to perform vehicle detection on the reference frame by using the vehicle detection model to obtain vehicles included in the reference frame;
  • the point and the pixel point corresponding to each pixel point in the reference frame respectively make differences on different color channels, and take the average value of the obtained difference values of the different color channels to obtain the difference value matrix to be processed;
  • the difference matrix to be processed the elements corresponding to the vehicle included in the reference frame and the at least one vehicle to be tested in the current frame are replaced by zero, and the absolute value of the replaced difference matrix is taken to obtain the processing the difference matrix; calculating the average value of the processed difference matrix to obtain a background difference value; when the background difference value is greater than a preset difference threshold, determine that the movement judgment result is movement; in the background When the difference value is not greater than the preset difference threshold, it is determined that the movement judgment result is no movement.
  • the splicing module 502 is specifically configured to, in the current frame, generate a set of detection candidate frames around each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one set of detection candidate frames candidate frame; from the reference frame, respectively select at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames; compare the at least one set of detection candidate frames with the at least one set of matching candidate frames frame, the candidate frames corresponding to the positions are spliced to obtain the at least one set of spliced images.
  • the splicing module 502 is specifically configured to generate a first candidate frame around a first vehicle in the current frame; the first vehicle is any one of the at least one vehicle to be tested vehicle to be tested; reduce at least a part of the candidate frames in the first candidate frame; select candidate frames that meet the preset size conditions from the reduced candidate frames, and candidate frames that are not reduced in the first candidate frame A group of detection candidate frames corresponding to the first vehicle is formed.
  • the splicing module 502 is specifically configured to reduce the candidate frame located under the first vehicle in the first candidate frame according to a preset reduction scale.
  • the splicing module 502 is specifically configured to reduce the first candidate frame to the candidate frame that overlaps with the second candidate frame generated around the second vehicle; wherein the second vehicle is the at least one candidate frame.
  • a vehicle to be tested that is different from the first vehicle in a vehicle to be tested.
  • the throwing event detection device further includes a training module (not shown in the figure);
  • the training module is configured to obtain vehicle detection samples and a preset detection model; perform vehicle detection training on the preset detection model by using the vehicle detection samples to obtain the vehicle detection model;
  • FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 6 , the electronic device includes: a processor 601, a memory 602 and a communication bus 603;
  • the communication bus 603 is configured to realize the communication connection between the processor 601 and the memory 602;
  • the processor 601 is configured to execute one or more programs stored in the memory 602, so as to implement the above-mentioned method for detecting a throwing event.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the above-mentioned Spill event detection method.
  • the computer-readable storage medium may be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read Only Memory) -Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); it can also be a respective device including one or any combination of the above memories, Such as mobile phones, computers, tablet devices, personal digital assistants, etc.
  • An embodiment of the present disclosure also provides a computer program product, the computer program product includes a computer program or an instruction, and when the computer program or instruction runs on a computer, the computer is made to execute the above method for detecting a throwing event.
  • embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable signal processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
  • Embodiments of the present disclosure provide a throwing event detection method, device, electronic device, storage medium and computer program product.
  • the method includes: using a vehicle detection model to perform vehicle detection on a current frame, and determining each detected vehicle as a Vehicles to be tested, at least one vehicle to be tested is obtained; a reference frame corresponding to the current frame is obtained, and for each vehicle to be tested in at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are spliced to obtain at least one vehicle to be tested.
  • At least one group of stitched images corresponding to a vehicle to be tested the same position is the position around the vehicle to be tested in the current frame and the reference frame; using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, based on at least one
  • a set of spliced images corresponding to the set of spliced images is subjected to throwing event detection, and at least one throwing event detection result is obtained.
  • the throwing event detection method of the present disclosure uses a vehicle detection model to first perform vehicle detection from an image, and then uses the throwing event detection model to detect the throwing event based on the stitched images corresponding to the vehicle, which not only associates the throwing event with a specific vehicle, but also improves the Improved spill event detection performance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

A spill-out event detection method and apparatus, an electronic device, a storage medium, and a computer program product. The method comprises: carrying out vehicle detection on a current frame by using a vehicle detection model, determining each detected vehicle as a vehicle to be tested, and obtaining at least one vehicle to be tested (S101); acquiring a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing images corresponding to the same position in the current frame and reference frame to obtain at least one group of spliced images corresponding to the at least one vehicle to be tested, the same position being the position around the vehicle to be tested in the current frame and reference frame (S102); and using a spill-out event detection model to carry out, on the basis of a corresponding group of spliced images in the at least one group of spliced images, spill-out event detection on each vehicle in the at least one vehicle to be tested, so as to obtain at least one spill-out event detection result (S103).

Description

抛洒事件检测方法、装置、电子设备、存储介质及计算机程序产品Throwing event detection method, device, electronic device, storage medium and computer program product
相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS
本申请基于申请号为202110483379.6、申请日为2021年04月30日,申请名称为“抛洒事件检测方法、装置、电子设备及存储介质”的中国专利申请提出,并要求该中国专利申请的优先权,该中国专利申请的全部内容在此以引入方式结合在本申请中。This application is based on the Chinese patent application with the application number of 202110483379.6 and the application date of April 30, 2021, and the application name is "throwing event detection method, device, electronic equipment and storage medium", and claims the priority of the Chinese patent application , the entire content of this Chinese patent application is hereby incorporated by reference into this application.
技术领域technical field
本公开涉及计算机视觉技术领域,尤其涉及一种抛洒事件检测方法、装置、电子设备、存储介质及计算机程序产品。The present disclosure relates to the field of computer vision technology, and in particular, to a method, device, electronic device, storage medium and computer program product for detecting a throwing event.
背景技术Background technique
在视频监控分析中,机动车异物抛洒事件检测是使用需求较高的分析事件。目前,对于机动车异物抛洒事件检测的方式,主要是基于目标检测算法,直接在视频图像中进行抛洒物的识别。In video surveillance analysis, the detection of motor vehicle foreign object throwing events is an analysis event with high usage requirements. At present, the detection method for the incident of vehicle foreign object throwing is mainly based on the target detection algorithm, and the throwing object is identified directly in the video image.
然而,基于目标检测算法的抛洒事件检测,仅能识别出图像中的抛洒物,检测得到的信息单一。However, the detection of spillage events based on the target detection algorithm can only identify the spilled objects in the image, and the detected information is single.
发明内容SUMMARY OF THE INVENTION
本公开实施例期望提供一种抛洒事件检测方法、装置、电子设备、存储介质及计算机程序产品。Embodiments of the present disclosure are expected to provide a method, apparatus, electronic device, storage medium, and computer program product for detecting a spillage event.
本公开实施例的技术方案是这样实现的:The technical solutions of the embodiments of the present disclosure are implemented as follows:
本公开实施例提供了一种抛洒事件检测方法,所述方法包括:Embodiments of the present disclosure provide a method for detecting a throwing event, the method comprising:
利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;Use the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested;
获取所述当前帧对应的参考帧,并针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像;所述同一位置为所述当前帧和所述参考帧中所述待测车辆周围的位置;Obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and an image corresponding to the same position in the reference frame to obtain the at least one vehicle to be tested at least one group of stitched images corresponding to the vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。Using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, throwing event detection is performed based on a corresponding set of stitched images in the at least one set of stitched images, to obtain at least one throwing event detection result.
在上述方法中,所述获取所述当前帧对应的参考帧,包括:In the above method, the obtaining the reference frame corresponding to the current frame includes:
获取采集到所述当前帧之前预设时长内采集到的至少一帧图像;acquiring at least one frame of image collected within a preset time period before the current frame is collected;
利用所述车辆检测模型,对所述至少一帧图像中每一帧图像分别进行车辆检测,并统计每一帧图像检测出的车辆数量;Using the vehicle detection model, vehicle detection is performed on each frame of the at least one frame of image, and the number of vehicles detected in each frame of image is counted;
将所述至少一帧图像中检测出的所述车辆数量最少的帧图像确定为所述参考帧。A frame image with the least number of vehicles detected in the at least one frame image is determined as the reference frame.
在上述方法中,所述利用抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测之前,所述方法还包括:In the above method, before using the throwing event detection model to detect the throwing event of each vehicle to be tested in the at least one vehicle to be tested based on the at least one set of stitched images, the method further includes:
利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果;Use the reference frame to judge the suspected throwing sample of the current frame to obtain a sample judgment result;
所述利用抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测,包括:The said using the throwing event detection model, based on the at least one group of stitched images, to perform throwing event detection on each vehicle to be tested in the at least one vehicle to be tested, including:
在所述样本判断结果为疑似抛洒样本的情况下,利用所述抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测。In the case that the sample judgment result is a suspected throwing sample, the throwing event detection model is used to perform throwing event detection for each vehicle to be tested in the at least one vehicle to be tested based on the at least one set of stitched images.
在上述方法中,所述利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果,包括:In the above method, the use of the reference frame to judge the suspected throwing sample of the current frame to obtain a sample judgment result, including:
将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差后取绝对值,并对得到的不同色彩通道的绝对差值取均值,得到第一差值矩阵;Each pixel in the current frame and the pixel at the position corresponding to each pixel in the reference frame are compared on different color channels to obtain the absolute value, and the obtained values of the different color channels are compared. The absolute difference is averaged to obtain the first difference matrix;
对所述第一差值矩阵进行低通滤波处理,得到第二差值矩阵;performing low-pass filtering processing on the first difference matrix to obtain a second difference matrix;
对所述第二差值矩阵取均值,得到目标评估均值;Taking the mean value of the second difference matrix to obtain the mean value of the target evaluation;
在所述目标评估均值大于预设均值阈值的情况下,确定所述样本判断结果为疑似抛洒样本;In the case that the target evaluation mean value is greater than the preset mean value threshold, it is determined that the sample judgment result is a suspected throwing sample;
在所述目标评估均值不大于所述预设均值阈值的情况下,确定所述样本判断结果为非疑似抛洒样本。In the case that the target evaluation mean is not greater than the preset mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
在上述方法中,所述利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果之前,所述方法还包括;In the above method, before the sample judgment is performed on the current frame by using the reference frame, and the sample judgment result is obtained, the method further includes;
基于所述当前帧和所述参考帧进行拍摄镜头移动判断,得到移动判断结果;Based on the current frame and the reference frame, the movement judgment of the shooting lens is performed to obtain a movement judgment result;
所述利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果,包括:The said reference frame is used to judge the suspected throwing sample of the current frame, and the sample judgment result is obtained, including:
在所述移动判断结果为未移动的情况下,利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到所述样本判断结果。In the case that the movement judgment result is no movement, the sample judgment result of the sample is obtained by using the reference frame to judge the suspected throwing sample of the current frame.
在上述方法中,所述基于所述当前帧和所述参考帧进行拍摄镜头移动判断,得到移动判断结果,包括:In the above method, the movement judgment of the shooting lens is performed based on the current frame and the reference frame to obtain a movement judgment result, including:
利用所述车辆检测模型对所述参考帧进行车辆检测,得到所述参考帧包括的车辆;Perform vehicle detection on the reference frame by using the vehicle detection model to obtain vehicles included in the reference frame;
将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差,并对得到的不同色彩通道的差值取均值,得到待处理差值矩阵;Differentiate each pixel in the current frame and the pixel at the corresponding position of each pixel in the reference frame on different color channels, and average the obtained differences in different color channels. , get the difference matrix to be processed;
将所述待处理差值矩阵中,所述参考帧包括的车辆和所述当前帧中所述至少一个待测车辆对应的元素替换为零,并对替换后的差值矩阵取绝对值,得到已处理差值矩阵;In the difference matrix to be processed, the vehicle included in the reference frame and the element corresponding to the at least one vehicle to be tested in the current frame are replaced by zero, and the absolute value of the replaced difference matrix is obtained to obtain Processed difference matrix;
计算所述已处理差值矩阵的平均值,得到背景差异值;Calculate the average value of the processed difference matrix to obtain the background difference value;
在所述背景差异值大于预设差异阈值的情况下,确定所述移动判断结果为移动;In the case that the background difference value is greater than a preset difference threshold, determine that the movement judgment result is movement;
在所述背景差异值不大于所述预设差异阈值的情况下,确定所述移动判断结果为未移动。In the case that the background difference value is not greater than the preset difference threshold, it is determined that the movement determination result is no movement.
在上述方法中,所述针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像,包括:In the above method, for each vehicle to be tested in the at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are spliced to obtain the corresponding image of the at least one vehicle to be tested. at least one set of stitched images, including:
在所述当前帧中,所述至少一个待测车辆的每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框;In the current frame, a set of detection candidate frames is generated around each vehicle to be tested of the at least one vehicle to be tested, to obtain at least one set of detection candidate frames;
从所述参考帧中,分别选取与所述至少一组检测候选框位置一一对应的至少一组匹配候选框;From the reference frame, respectively select at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames;
将所述至少一组检测候选框与所述至少一组匹配候选框中,位置对应的候选框进行拼接,得到所述至少一组拼接图像。The at least one set of detection candidate frames and the candidate frames corresponding to the positions of the at least one set of matching candidate frames are spliced to obtain the at least one set of spliced images.
在上述方法中,所述在所述当前帧中,所述至少一个待测车辆的每个待测车辆周围 生成一组检测候选框,得到至少一组检测候选框,包括:In the above method, in the current frame, a set of detection candidate frames is generated around each vehicle to be tested of the at least one vehicle to be tested, to obtain at least one set of detection candidate frames, including:
在所述当前帧中,第一车辆周围生成第一候选框;所述第一车辆为所述至少一个待测车辆中任一待测车辆;In the current frame, a first candidate frame is generated around the first vehicle; the first vehicle is any vehicle to be tested among the at least one vehicle to be tested;
对所述第一候选框中至少一部分候选框进行缩减;reducing at least a part of the candidate frames in the first candidate frame;
从缩减后的候选框中选取出满足预设尺寸条件的候选框,与所述第一候选框中未缩减的候选框组成所述第一车辆对应的一组检测候选框。A candidate frame that satisfies a preset size condition is selected from the reduced candidate frame, and the candidate frame that is not reduced in the first candidate frame forms a group of detection candidate frames corresponding to the first vehicle.
在上述方法中,所述对所述第一候选框中至少一部分候选框进行缩减,包括:In the above method, the reducing at least a part of the candidate frames in the first candidate frame includes:
对所述第一候选框中,位于所述第一车辆下方的候选框,按照预设缩减尺度进行缩减。For the first candidate frame, the candidate frame located below the first vehicle is reduced according to a preset reduction scale.
在上述方法中,所述对所述第一候选框中至少一部分候选框进行缩减,包括:In the above method, the reducing at least a part of the candidate frames in the first candidate frame includes:
缩减所述第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,所述第二车辆为所述至少一个待测车辆中与所述第一车辆不同的待测车辆。Reduce the first candidate frame to a candidate frame that overlaps with the second candidate frame generated around the second vehicle; wherein the second vehicle is the at least one vehicle to be tested that is different from the first vehicle. test vehicle.
在上述方法中,所述利用车辆检测模型对当前帧进行车辆检测之前,所述方法还包括:In the above method, before the vehicle detection is performed on the current frame by using the vehicle detection model, the method further includes:
获取车辆检测样本和预设检测模型;Obtain vehicle detection samples and preset detection models;
利用所述车辆检测样本对所述预设检测模型进行车辆检测训练,得到所述车辆检测模型;Use the vehicle detection samples to perform vehicle detection training on the preset detection model to obtain the vehicle detection model;
所述利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测之前,所述方法还包括:Before using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, the method further includes:
获取车辆抛洒样本和预设时序差值神经网络;Obtain vehicle throwing samples and preset time series difference neural network;
利用所述车辆抛洒样本对所述预设时序差值神经网络进行车辆抛洒事件检测训练,得到所述抛洒事件检测模型。Using the vehicle throwing samples to perform vehicle throwing event detection training on the preset time sequence difference neural network to obtain the throwing event detection model.
本公开实施例提供了一种抛洒事件检测装置,包括:The embodiment of the present disclosure provides a throwing event detection device, including:
检测模块,配置为利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;a detection module, configured to perform vehicle detection on the current frame by using a vehicle detection model, and determine each detected vehicle as a vehicle to be tested, to obtain at least one vehicle to be tested;
拼接模块,配置为获取所述当前帧对应的参考帧,并针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像;所述同一位置为所述当前帧和所述参考帧中所述待测车辆周围的位置;a splicing module, configured to obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and an image corresponding to the same position in the reference frame, obtaining at least one group of stitched images corresponding to the at least one vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
所述检测模块,还配置为利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。The detection module is further configured to use a throwing event detection model to detect a throwing event based on a corresponding set of stitched images in the at least one set of stitched images for each vehicle to be tested in the at least one vehicle to be tested, and obtain: At least one spill event detection result.
在上述装置中,所述拼接模块,具体配置为获取采集到所述当前帧之前预设时长内采集到的至少一帧图像;利用所述车辆检测模型,对所述至少一帧图像中每一帧图像分别进行车辆检测,并统计所述每一帧图像检测出的车辆数量;将所述至少一帧图像中检测出的所述车辆数量最少的帧图像确定为所述参考帧。In the above device, the splicing module is specifically configured to acquire at least one frame of image collected within a preset time period before the current frame is collected; using the vehicle detection model, for each image in the at least one frame of image Vehicle detection is performed on the frame images respectively, and the number of vehicles detected in each frame image is counted; the frame image with the least number of vehicles detected in the at least one frame image is determined as the reference frame.
在上述装置中,所述检测模块,还配置为利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果;In the above device, the detection module is further configured to use the reference frame to perform a sample judgment on the current frame suspected to be thrown, to obtain a sample judgment result;
所述检测模块,具体配置为在所述样本判断结果为疑似抛洒样本的情况下,利用所述抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测。The detection module is specifically configured to, when the sample judgment result is a suspected throwing sample, use the throwing event detection model, based on the at least one set of stitched images, to detect each of the at least one vehicle to be tested. Test vehicle for spill event detection.
在上述装置中,所述检测模块,具体配置为将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差后取绝对值,并对得到的不同色彩通道的绝对差值取均值,得到第一差值矩阵;对所述第一差值矩阵进 行低通滤波处理,得到第二差值矩阵;对所述第二差值矩阵取均值,得到目标评估均值;在所述目标评估均值大于预设均值阈值的情况下,确定所述样本判断结果为疑似抛洒样本;在所述目标评估均值不大于所述预设均值阈值的情况下,确定所述样本判断结果为非疑似抛洒样本。In the above device, the detection module is specifically configured to make a difference between each pixel in the current frame and a pixel at a position corresponding to each pixel in the reference frame on different color channels. Then take the absolute value, and take the mean value of the obtained absolute difference values of different color channels to obtain a first difference value matrix; perform low-pass filtering on the first difference value matrix to obtain a second difference value matrix; The second difference matrix takes the mean value to obtain the target evaluation mean value; in the case that the target evaluation mean value is greater than the preset mean value threshold, it is determined that the sample judgment result is a suspected throwing sample; when the target evaluation mean value is not greater than the predetermined mean value threshold In the case of setting the mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
在上述装置中,所述检测模块,还配置为基于所述当前帧和所述参考帧进行拍摄镜头移动判断,得到移动判断结果;In the above device, the detection module is further configured to judge the movement of the shooting lens based on the current frame and the reference frame, and obtain a movement judgment result;
所述检测模块,具体配置为在所述移动判断结果为未移动的情况下,利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到所述样本判断结果。The detection module is specifically configured to use the reference frame to judge a suspected throwing sample of the current frame when the movement judgment result is no movement, to obtain the sample judgment result.
在上述装置中,所述检测模块,具体配置为利用所述车辆检测模型对所述参考帧进行车辆检测,得到所述参考帧包括的车辆;将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差,并对得到的不同色彩通道的差值取均值,得到待处理差值矩阵;将所述待处理差值矩阵中,所述参考帧包括的车辆和所述当前帧中所述至少一个待测车辆对应的元素替换为零,并对替换后的差值矩阵取绝对值,得到已处理差值矩阵;计算所述已处理差值矩阵的平均值,得到背景差异值;在所述背景差异值大于预设差异阈值的情况下,确定所述移动判断结果为移动;在所述背景差异值不大于所述预设差异阈值的情况下,确定所述移动判断结果为未移动。In the above device, the detection module is specifically configured to use the vehicle detection model to perform vehicle detection on the reference frame to obtain vehicles included in the reference frame; and compare each pixel in the current frame with the reference frame. The pixel points in the reference frame corresponding to the positions of each pixel point are respectively different in different color channels, and the difference values of the obtained different color channels are averaged to obtain a difference matrix to be processed; In the difference matrix, the element corresponding to the vehicle included in the reference frame and the at least one vehicle to be tested in the current frame is replaced by zero, and the absolute value of the replaced difference matrix is taken to obtain the processed difference matrix. Calculate the average value of the processed difference matrix to obtain the background difference value; in the case that the background difference value is greater than the preset difference threshold value, determine that the movement judgment result is movement; when the background difference value is not greater than In the case of the preset difference threshold, it is determined that the movement determination result is no movement.
在上述装置中,所述拼接模块,具体配置为在所述当前帧中,所述至少一个待测车辆中每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框;从所述参考帧中,分别选取与所述至少一组检测候选框位置一一对应的至少一组匹配候选框;将所述至少一组检测候选框与所述至少一组匹配候选框中,位置对应的候选框进行拼接,得到所述至少一组拼接图像。In the above device, the splicing module is specifically configured to, in the current frame, generate a set of detection candidate frames around each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one set of detection candidate frames; In the reference frame, at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames are selected respectively; The corresponding candidate frames are spliced to obtain the at least one set of spliced images.
在上述装置中,所述拼接模块,具体配置为在所述当前帧中,第一车辆周围生成第一候选框;所述第一车辆为所述至少一个待测车辆中任一待测车辆;对所述第一候选框中至少一部分候选框进行缩减;从缩减后的候选框中选取出满足预设尺寸条件的候选框,与所述第一候选框中未缩减的候选框组成所述第一车辆对应的一组检测候选框。In the above device, the splicing module is specifically configured to generate a first candidate frame around a first vehicle in the current frame; the first vehicle is any vehicle to be tested among the at least one vehicle to be tested; Reduce at least a part of the candidate frames in the first candidate frame; select candidate frames that meet the preset size condition from the reduced candidate frames, and form the first candidate frame with the unreduced candidate frames in the first candidate frame. A set of detection candidate boxes corresponding to a vehicle.
在上述装置中,所述拼接模块,具体配置为对所述第一候选框中,位于所述第一车辆下方的候选框,按照预设缩减尺度进行缩减。In the above device, the splicing module is specifically configured to reduce the first candidate frame and the candidate frame below the first vehicle according to a preset reduction scale.
在上述装置中,所述拼接模块,具体配置为缩减所述第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,所述第二车辆为所述至少一个待测车辆中与所述第一车辆不同的待测车辆。In the above device, the splicing module is specifically configured to reduce the first candidate frame to a candidate frame that overlaps with a second candidate frame generated around a second vehicle; wherein the second vehicle is the at least one A vehicle to be tested that is different from the first vehicle among the vehicles to be tested.
在上述装置中,还包括训练模块;In the above device, also includes a training module;
所述训练模块,配置为获取车辆检测样本和预设检测模型;利用所述车辆检测样本对所述预设检测模型进行车辆检测训练,得到所述车辆检测模型;The training module is configured to obtain vehicle detection samples and a preset detection model; perform vehicle detection training on the preset detection model by using the vehicle detection samples to obtain the vehicle detection model;
以及,获取车辆抛洒样本和预设时序差值神经网络;利用所述车辆抛洒样本对所述预设时序差值神经网络进行车辆抛洒事件检测训练,得到所述抛洒事件检测模型。And, obtaining vehicle throwing samples and a preset timing difference neural network; using the vehicle throwing samples to perform vehicle throwing event detection training on the preset timing difference neural network to obtain the throwing event detection model.
本公开实施例提供了一种电子设备,包括:处理器、存储器和通信总线;Embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a communication bus;
所述通信总线,配置为实现所述处理器和所述存储器之间的通信连接;the communication bus configured to implement a communication connection between the processor and the memory;
所述处理器,配置为执行所述存储器中存储的一个或多个程序,以实现上述抛洒事件检测方法。The processor is configured to execute one or more programs stored in the memory, so as to implement the above method for detecting a throwing event.
本公开实施例提供了一种计算机可读存储介质,所述计算机可读存储介质存储有一个或者多个程序,所述一个或者多个程序可以被一个或者多个处理器执行,以实现上述抛洒事件检测方法。Embodiments of the present disclosure provide a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the above-mentioned throwing Event detection method.
本公开实施例提供了一种计算机程序产品,所述计算机程序产品包括计算机程序或指令,在所述计算机程序或指令在计算机上运行的情况下,使所述计算机执行上述抛洒 事件检测方法。An embodiment of the present disclosure provides a computer program product, the computer program product includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the above method for detecting a spillage event.
本公开实施例提供了一种抛洒事件检测方法、装置、电子设备、存储介质及计算机程序产品,方法包括:利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;获取当前帧对应的参考帧,并针对至少一个待测车辆中每个待测车辆,将当前帧和参考帧中对应同一位置的图像进行拼接,得到至少一个待测车辆对应的至少一组拼接图像;同一位置为当前帧和参考帧中待测车辆周围的位置;利用抛洒事件检测模型,针对至少一个待测车辆中每个待测车辆,基于至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。本公开的抛洒事件检测方法,采用车辆检测模型先从图像中进行车辆检测,再利用抛洒事件检测模型,基于车辆对应的拼接图像进行抛洒事件检测,不仅将抛洒事件与具体车辆关联起来,还提高了抛洒事件检测性能。Embodiments of the present disclosure provide a throwing event detection method, device, electronic device, storage medium and computer program product. The method includes: using a vehicle detection model to perform vehicle detection on a current frame, and determining each detected vehicle as a Vehicles to be tested, at least one vehicle to be tested is obtained; a reference frame corresponding to the current frame is obtained, and for each vehicle to be tested in at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are spliced to obtain at least one vehicle to be tested. At least one group of stitched images corresponding to a vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame; using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, based on at least one A set of spliced images corresponding to the set of spliced images is subjected to throwing event detection, and at least one throwing event detection result is obtained. The throwing event detection method of the present disclosure uses a vehicle detection model to first perform vehicle detection from an image, and then uses the throwing event detection model to detect the throwing event based on the stitched images corresponding to the vehicle, which not only associates the throwing event with a specific vehicle, but also improves the Improved spill event detection performance.
附图说明Description of drawings
图1为本公开实施例提供的一种抛洒事件检测方法的流程示意图;1 is a schematic flowchart of a method for detecting a throwing event according to an embodiment of the present disclosure;
图2为本公开实施例提供的一种示例性的预设检测模型的结构示意图;FIG. 2 is a schematic structural diagram of an exemplary preset detection model provided by an embodiment of the present disclosure;
图3为本公开实施例提供的一种示例性的第一候选框示意图;FIG. 3 is a schematic diagram of an exemplary first candidate frame provided by an embodiment of the present disclosure;
图4为本公开实施例提供的一种示例性的拼接图像;FIG. 4 provides an exemplary mosaic image according to an embodiment of the present disclosure;
图5为本公开实施例提供的一种抛洒事件检测装置的结构示意图;5 is a schematic structural diagram of a throwing event detection device according to an embodiment of the present disclosure;
图6为本公开实施例提供的一种电子设备的结构示意图。FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
具体实施方式Detailed ways
下面将结合本公开实施例中的附图,对本公开实施例中的技术方案进行清楚、完整地描述。The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure.
本公开实施例提供了一种抛洒事件检测方法,其执行主体可以是抛洒事件检测装置,例如,抛洒事件检测方法可以由终端设备或服务器或其它电子设备执行,其中,终端设备可以为用户设备(User Equipment,UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant,PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中,抛洒事件检测方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。Embodiments of the present disclosure provide a method for detecting a spillage event, the execution body of which may be a device for detecting a spillage event. For example, the method for detecting a spillage event may be executed by a terminal device or a server or other electronic device, where the terminal device may be a user equipment ( User Equipment, UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (Personal Digital Assistant, PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. In some possible implementations, the throwing event detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.
图1为本公开实施例提供的一种抛洒事件检测方法的流程示意图。如图1所示,抛洒事件检测方法主要包括以下步骤:FIG. 1 is a schematic flowchart of a method for detecting a throwing event according to an embodiment of the present disclosure. As shown in Figure 1, the method for detecting a spillage event mainly includes the following steps:
S101、利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为待测车辆,得到至少一个待测车辆。S101. Use a vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested.
在本公开的实施例中,抛洒事件检测装置在获取到当前帧的情况下,可以利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为待测车辆,得到至少一个待测车辆。In the embodiment of the present disclosure, when the throwing event detection device obtains the current frame, it can use the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested, and obtain at least one vehicle to be tested.
需要说明的是,在本公开的实施例中,抛洒事件检测装置可以包括拍摄镜头,用于实现当前帧的获取,当前帧为当前需要进行抛洒事件检测的帧图像,可以是当前采集到的。此外,拍摄镜头也与抛洒事件检测装置为两个独立的设备,由拍摄镜头将采集到的当前帧传输至抛洒事件检测模型。具体的当前帧的来源本公开实施例不作限定。It should be noted that, in the embodiment of the present disclosure, the throwing event detection apparatus may include a photographing lens for obtaining a current frame, which is a frame image currently required for throwing event detection, which may be currently collected. In addition, the shooting lens and the throwing event detection device are two independent devices, and the shooting lens transmits the collected current frame to the throwing event detection model. The specific source of the current frame is not limited in this embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置中存储有车辆检测模型,其可以对输入的图像进行车辆检测,识别出输入的图像中的车辆。具体的车辆检测模型本公开实施例不作限定。It should be noted that, in the embodiment of the present disclosure, a vehicle detection model is stored in the throwing event detection device, which can perform vehicle detection on the input image and identify the vehicle in the input image. The specific vehicle detection model is not limited in the embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置将当前帧输入车辆检测模块,车辆检测模型可以从当前帧中,针对每一个车辆定位出一个车辆检测框,每个车辆检测框中包含了一个车辆的图像。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device inputs the current frame into the vehicle detection module, and the vehicle detection model can locate a vehicle detection frame for each vehicle from the current frame, and each vehicle detection frame contains an image of a vehicle.
需要说明的是,在本公开的实施例中,抛洒事件检测装置在利用车辆检测模型对当前帧进行车辆检测之前,还可以执行以下步骤:获取车辆检测样本和预设检测模型;利用车辆检测样本对预设检测模型进行车辆检测训练,得到车辆检测模型。It should be noted that, in the embodiment of the present disclosure, before using the vehicle detection model to perform vehicle detection on the current frame, the throwing event detection device may further perform the following steps: acquiring a vehicle detection sample and a preset detection model; using the vehicle detection sample Perform vehicle detection training on the preset detection model to obtain a vehicle detection model.
可以理解的是,在本公开的实施例中,抛洒事件检测装置获取车辆检测样本,用于进行预设检测模型的训练,从而可以得到精度较高的车辆检测模型,保证车辆检测的准确性。It can be understood that, in the embodiment of the present disclosure, the throwing event detection device obtains vehicle detection samples for training a preset detection model, thereby obtaining a vehicle detection model with higher accuracy and ensuring the accuracy of vehicle detection.
可以理解的是,在本公开的实施例中,车辆检测模型可以是抛洒事件检测装置利用车辆检测样本对预设检测模型训练后得到的,具体的车辆检测样本和预设检测模型可以根据实际需求设置和选择,本公开实施例不作限定。It can be understood that, in the embodiment of the present disclosure, the vehicle detection model may be obtained after training the preset detection model by the vehicle detection sample by the throwing event detection device, and the specific vehicle detection sample and the preset detection model can be based on actual needs. The settings and selections are not limited in the embodiments of the present disclosure.
图2为本公开实施例提供的一种示例性的预设检测模型的结构示意图。如图2所示,预设检测模型可以包括卷积层、池化层和全连接层等。抛洒事件检测装置可以将车辆检测样本输入预设检测模型,通过不同处理层依次进行处理,从而得到最终的车辆检测结果与预设的标准检测结果进行比较,从而调整预设检测模型中的相关参数,最终得到车辆检测模型。FIG. 2 is a schematic structural diagram of an exemplary preset detection model provided by an embodiment of the present disclosure. As shown in Figure 2, the preset detection model may include convolutional layers, pooling layers, and fully connected layers. The throwing event detection device can input the vehicle detection samples into the preset detection model, and process them in sequence through different processing layers, so as to obtain the final vehicle detection result and compare the preset standard detection result, thereby adjusting the relevant parameters in the preset detection model , and finally get the vehicle detection model.
需要说明的是,在本申请的实施例中,抛洒事件检测装置将当前帧输入车辆检测模型,从而从车辆检测模型输出从当前帧中检测出的每一个车辆,即为待测车辆。具体的待测车辆的数量本公开实施例不作限定。It should be noted that, in the embodiment of the present application, the throwing event detection device inputs the current frame into the vehicle detection model, so that each vehicle detected from the current frame is output from the vehicle detection model, that is, the vehicle to be tested. The specific number of vehicles to be tested is not limited in the embodiment of the present disclosure.
S102、获取当前帧对应的参考帧,并针对至少一个待测车辆中每个待测车辆,将当前帧和参考帧对应同一位置的图像进行拼接,得到至少一个待测车辆对应的至少一组拼接图像;同一位置为当前帧和参考帧中待测车辆周围的位置。S102: Acquire a reference frame corresponding to the current frame, and for each vehicle to be tested in at least one vehicle to be tested, splicing images corresponding to the same position of the current frame and the reference frame to obtain at least one set of splices corresponding to at least one vehicle to be tested Image; the same position is the position around the vehicle under test in the current frame and the reference frame.
在本申请的实施例中,抛洒事件检测装置在获取到当前帧的情况下,可以获取当前帧对应的参考帧,从而针对至少一个待测车辆中每个待测车辆,将当前帧和参考帧对应同一位置的图像进行拼接,得到至少一个待测车辆对应的至少一组拼接图像。In the embodiment of the present application, the throwing event detection device can obtain the reference frame corresponding to the current frame when the current frame is obtained, so as to compare the current frame and the reference frame for each vehicle to be tested in the at least one vehicle to be tested Images corresponding to the same position are stitched to obtain at least one set of stitched images corresponding to at least one vehicle to be tested.
具体的,在本公开的实施例中,抛洒事件检测装置获取当前帧对应的参考帧,包括:获取采集到当前帧之间预设时长内采集到的至少一帧图像;利用车辆检测模型,对至少一帧图像中每一帧图像分别进行车辆检测,并统计每一帧图像检测出的车辆数量;将至少一帧图像中检测出的车辆数量最少的帧图像确定为参考帧。Specifically, in the embodiment of the present disclosure, the throwing event detection device obtains a reference frame corresponding to the current frame, including: obtaining at least one frame of images collected within a preset time period between the current frames; using a vehicle detection model to detect Vehicle detection is performed for each frame of image in the at least one frame of image, and the number of vehicles detected in each frame of image is counted; the frame image with the least number of vehicles detected in the at least one frame of image is determined as a reference frame.
可以理解的是,在本公开的实施例中,抛洒事件检测装置中存储有预设时长,抛洒事件检测装置可以以采集到当前帧的时刻为基准,获取该时刻之间预设时长内采集到的至少一帧图像。具体的预设时长可以根据实际需求和应用场景设定,本公开实施例不作限定。It can be understood that, in the embodiment of the present disclosure, a preset time period is stored in the throwing event detection device, and the throwing event detection device may take the moment when the current frame is collected as a reference, and obtain the data collected within the preset duration between the moments. at least one frame of image. The specific preset duration may be set according to actual requirements and application scenarios, which is not limited in the embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置可以将至少一帧图像中每一帧图像分别输入车辆检测模型,进行车辆检测,从而统计每一帧图像包括的车辆数量,进一步的,将其中车辆数量最少的帧图像作为当前帧对应的参考帧,这是因为,采用车辆数量最少的帧图像作为参考帧,可以最大限度的减少后续进行检测的干扰,避免大量车辆对抛洒物的遮挡,导致无法正确检测抛洒事件等问题。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device may input each frame of image in at least one frame of image into the vehicle detection model, respectively, to perform vehicle detection, so as to count the number of vehicles included in each frame of image, and further The frame image with the least number of vehicles is used as the reference frame corresponding to the current frame. This is because using the frame image with the least number of vehicles as the reference frame can minimize the interference of subsequent detection and avoid a large number of vehicles. occlusion, resulting in problems such as inability to correctly detect throwing events.
需要说明的是,在本公开的实施例中,抛洒事件检测装置可以采用上述方式获取当前帧对应参考帧,还可以采用其它方式确定参考帧,例如,将当前帧的前一帧图像确定为参考帧。具体的获取当前帧对应的参考帧的方式可以根据实际需求和应用场景设定,本公开实施例不作限定。It should be noted that, in the embodiments of the present disclosure, the throwing event detection apparatus may obtain the reference frame corresponding to the current frame in the above-mentioned manner, and may also determine the reference frame in other manners, for example, determining the image of the previous frame of the current frame as the reference frame frame. The specific manner of acquiring the reference frame corresponding to the current frame may be set according to actual requirements and application scenarios, which is not limited in the embodiment of the present disclosure.
具体的,在本公开的实施例中,抛洒事件检测装置针对至少一个待测车辆中每个待 测车辆,将当前帧和参考帧中对应同一位置的图像进行拼接,得到至少一个待测车辆对应的至少一组图像,包括:在当前帧中,至少一个待测车辆中每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框;从参考帧中,分别选取与至少一组检测候选框一一对应的至少一组匹配候选框;将至少一组检测候选框与至少一组匹配候选框中,位置对应的候选框进行拼接,得到至少一组拼接图像。Specifically, in the embodiment of the present disclosure, the throwing event detection device splices images corresponding to the same position in the current frame and the reference frame for each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one vehicle to be tested corresponding to at least one set of images, including: in the current frame, generating a set of detection candidate frames around each vehicle to be tested in at least one vehicle to be tested, to obtain at least one set of detection candidate frames; At least one set of matching candidate frames corresponding to one set of detection candidate frames; at least one set of detection candidate frames and at least one set of matching candidate frames are spliced with corresponding candidate frames to obtain at least one set of stitched images.
需要说明的是,在本公开的实施例中,抛洒事件检测装置在当前帧中,至少一个待测车辆中每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框,包括:在当前帧中,第一车辆周围生成第一候选框;第一车辆为至少一个待测车辆中任一待测车辆;对第一候选框中至少一部分候选框进行缩减;从缩减后的候选框中选取出满足预设尺寸条件的候选框,与第一候选框中未缩减的候选框组成第一车辆对应的一组检测候选框。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device generates a set of detection candidate frames around each vehicle to be tested in at least one vehicle to be tested in the current frame, and obtains at least one set of candidate detection frames, including : In the current frame, a first candidate frame is generated around the first vehicle; the first vehicle is any vehicle to be tested in at least one vehicle to be tested; at least a part of the candidate frames in the first candidate frame is reduced; A candidate frame that satisfies the preset size condition is selected from the frame, and the candidate frame that is not reduced in the first candidate frame forms a set of detection candidate frames corresponding to the first vehicle.
需要说明的是,在本公开的实施例中,抛洒事件检测装置在当前帧中,第一车辆周围生成第一候选框,实际上就是在包含第一车辆的车辆检测框周围生成第一候选框。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device generates a first candidate frame around the first vehicle in the current frame, which is actually generating a first candidate frame around the vehicle detection frame including the first vehicle .
图3为本公开实施例提供的一种示例性的第一候选框示意图。如图3所示,在本公开的实施例中,抛洒事件检测装置可以利用车辆检测模型,从当前帧中中检测出第一车辆,从而定位出包含第一车辆的车辆检测框,该车辆检测框为一个矩形框,抛洒事件检测装置具体在车辆检测框的周围生成12个第一候选框,每个第一候选框也为一个矩形框,其中,位于车辆检测框两侧的第一候选框,宽为车辆检测框的宽的一半,位于车辆检测框上下的第一候选框,长为车辆检测框的长的一半。具体的第一候选框的数量和尺寸可以根据实际需求设定,本公开实施例不作限定。FIG. 3 is a schematic diagram of an exemplary first candidate frame provided by an embodiment of the present disclosure. As shown in FIG. 3 , in the embodiment of the present disclosure, the device for detecting a throwing event can use a vehicle detection model to detect the first vehicle from the current frame, so as to locate a vehicle detection frame including the first vehicle, and the vehicle detects the first vehicle. The frame is a rectangular frame, and the throwing event detection device specifically generates 12 first candidate frames around the vehicle detection frame, and each first candidate frame is also a rectangular frame, wherein the first candidate frames located on both sides of the vehicle detection frame , the width is half the width of the vehicle detection frame, the first candidate frame located above and below the vehicle detection frame, and the length is half the length of the vehicle detection frame. The specific number and size of the first candidate frame may be set according to actual requirements, which is not limited in the embodiment of the present disclosure.
可以理解的是,在本公开的实施例中,由于在实际进行抛洒事件检测的场景中,检测频率较高,并且,距离待测车辆越远的位置,即使检测到抛洒事件,与待测车辆的关联性也很小,即,很大程度上并非是待测车辆对应的抛洒事件,因此,抛洒事件检测装置仅在包含第一车辆的车辆检测框的周围,按照上述尺寸生成一定数量的候选框,这样,在后续进行抛洒事件检测时,不仅检测效率较高,而且可以避免误检测。It can be understood that, in the embodiment of the present disclosure, since the detection frequency is relatively high in the actual scene of detection of the throwing event, and the farther from the vehicle to be tested, even if the throwing event is detected, it is different from the vehicle to be tested. The correlation is also very small, that is, it is largely not the throwing event corresponding to the vehicle to be tested. Therefore, the throwing event detection device only generates a certain number of candidates according to the above size around the vehicle detection frame containing the first vehicle. In this way, in the subsequent detection of the throwing event, not only the detection efficiency is high, but also false detection can be avoided.
具体的,在本公开的实施例中,抛洒事件检测装置对第一候选框中至少一部分候选框进行缩减,包括:对第一候选框中,位于第一车辆下方的候选框,按照预设缩减尺度进行缩减。Specifically, in the embodiment of the present disclosure, the throwing event detection device reduces at least a part of the candidate frames in the first candidate frame, including: reducing the candidate frames located under the first vehicle in the first candidate frame according to a preset scale down.
需要说明的是,在本公开的实施例中,抛洒事件检测装置对第一候选框中,位于第一车辆下方的候选框,按照预设缩减尺度进行缩减,可以是对第一候选框中,位于包含第一车辆的车辆检测框下方的候选框,长宽分别缩减10%,其形状不变。当然,具体的缩减程度可以根据实际需求和应用场景设定,本公开实施例不作限定。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device reduces the first candidate frame, the candidate frame located under the first vehicle, according to the preset reduction scale, which may be the first candidate frame, The candidate frame below the vehicle detection frame containing the first vehicle is reduced in length and width by 10%, and its shape remains unchanged. Of course, the specific reduction degree may be set according to actual requirements and application scenarios, which is not limited in the embodiment of the present disclosure.
可以理解的是,在本公开的实施例中,包含第一车辆的车辆检测框,其底部实际上已经距离第一车辆车体底部存在一定距离,而距离车辆检测框较远的位置进行抛洒事件检测,实际上与第一车辆之间不具备关联性,因此,抛洒事件检测装置可以适当的将车辆检测框下方的候选框进行尺寸缩减,从而避免误检。It can be understood that, in the embodiment of the present disclosure, the bottom of the vehicle detection frame including the first vehicle has a certain distance from the bottom of the body of the first vehicle, and the throwing event is performed at a position far away from the vehicle detection frame. The detection actually has no correlation with the first vehicle. Therefore, the throwing event detection device can appropriately reduce the size of the candidate frame below the vehicle detection frame, thereby avoiding false detection.
具体的,在本公开的实施例中,抛洒事件检测装置对第一候选框中至少一部分候选框进行缩减,包括:缩减第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,第二车辆为至少一个待测车辆中与第一车辆不同的待测车辆。Specifically, in the embodiment of the present disclosure, the throwing event detection device reduces at least a part of the candidate frames in the first candidate frame, including: reducing the first candidate frame, and the second candidate frame overlapped with the second candidate frame generated around the second vehicle Candidate frame; wherein, the second vehicle is a vehicle to be tested that is different from the first vehicle in at least one vehicle to be tested.
可以理解的是,在本公开的实施例中,对于不同待测车辆周围,也就是包含不同待测车辆的车辆检测框周围,抛洒事件检测装置均可以在周围生成对应的候选框,不同的车辆检测框周围的候选框可能存在重叠,在重叠区域检测到抛洒事件,实际上是难以准确区分对应的待测车辆,因此,抛洒事件检测装置缩减第一候选框中,与其它待测车辆周围生成的候选框重叠的候选框,从而后续进行检测可以避免误检,得到最优的检测结 果。It can be understood that, in the embodiment of the present disclosure, for the surroundings of different vehicles to be tested, that is, around the detection frames of vehicles including different vehicles to be tested, the throwing event detection device can generate corresponding candidate frames around them. The candidate frames around the detection frame may overlap, and when a throwing event is detected in the overlapping area, it is actually difficult to accurately distinguish the corresponding vehicle to be tested. Therefore, the throwing event detection device reduces the first candidate frame and generates it with the surrounding of other vehicles to be tested. The candidate frame overlaps the candidate frame, so that subsequent detection can avoid false detection and obtain the optimal detection result.
具体的,在本公开的实施例中,在本公开的实施例中,抛洒事件检测装置对第一候选框中至少一部分候选框进行缩减,还可以包括:对第一候选框中,位于第一车辆下方的候选框,按照预设缩减尺度进行缩减,以及,缩减第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,第二车辆为至少一个待测车辆中与第一车辆不同的待测车辆。Specifically, in the embodiment of the present disclosure, in the embodiment of the present disclosure, the throwing event detection apparatus reduces at least a part of the candidate frames in the first candidate frame, and may further include: The candidate frame below the vehicle is reduced according to the preset reduction scale, and the first candidate frame is reduced to a candidate frame that overlaps with the second candidate frame generated around the second vehicle; wherein the second vehicle is at least one vehicle to be tested A vehicle to be tested that is different from the first vehicle.
可以理解的是,在本公开的实施例中,抛洒事件检测装置可以采用上述两种缩减候选框的方式中任意一种,进行候选框缩减,当然,抛洒事件检测装置还可以同时采用上述两种缩减候选框的方式进行候选框缩减,本公开实施例不作限定。It can be understood that, in the embodiment of the present disclosure, the throwing event detection device can use any one of the above two methods to reduce the candidate frame to reduce the candidate frame. Of course, the throwing event detection device can also use the above two methods at the same time. The candidate frame reduction is performed by reducing the candidate frame, which is not limited in the embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置从缩减后的候选框中,选取出尺寸满足预设尺寸条件的每一个候选框,与第一候选框中未缩减的候选框组成第一车辆对应的一组检测候选框。具体的预设尺寸条件,可以包括预设长度、预设宽度,以及预设长宽比等,本公开实施例不作限定。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device selects each candidate frame whose size satisfies the preset size condition from the reduced candidate frame, and the candidate frame that is not reduced in the first candidate frame. A set of detection candidate boxes corresponding to the first vehicle is formed. The specific preset size conditions may include preset length, preset width, preset aspect ratio, etc., which are not limited in the embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置对于至少一个待测车辆中每个待测车辆均生成一组检测候选框,针对于每一组检测候选框中每个候选框,均可以从参考帧中获取位置对应的候选框,从而组成对应的一组匹配候选框。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device generates a set of detection candidate frames for each vehicle to be tested in at least one vehicle to be tested, and for each candidate frame of each set of detection candidate frames , the candidate frame corresponding to the position can be obtained from the reference frame, so as to form a corresponding set of matching candidate frames.
需要说明的是,在本公开的实施例中,抛洒事件检测装置在获得每个待测车辆对应的一组检测候选框和一组匹配候选框之后,对于每个待测车辆,可以将其对应的一组检测候选框中每个候选框,与其对应的一组匹配候选框中位置对应的候选框拼接,得到该位置相应的一个拼接图像。It should be noted that, in the embodiment of the present disclosure, after obtaining a set of detection candidate frames and a set of matching candidate frames corresponding to each vehicle to be tested, the device for detection of the throwing event may correspond to each vehicle to be tested. Each candidate frame in a set of detection candidate frames is spliced with the candidate frame corresponding to the position in the corresponding set of matching candidate frames, and a spliced image corresponding to the position is obtained.
图4为本公开实施例提供的一种示例性的拼接图像。如图4所示,该拼接图像左右分别为当前帧和参考帧中同一位置的候选框,两者可以实现拼接,从而得到拼接图像。FIG. 4 is an exemplary mosaic image provided by an embodiment of the present disclosure. As shown in Figure 4, the left and right sides of the spliced image are candidate frames at the same position in the current frame and the reference frame respectively, and splicing can be achieved between the two to obtain a spliced image.
S103、利用抛洒事件检测模型,针对至少一个待测车辆中每个待测车辆,基于至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。S103 . Using a throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, perform throwing event detection based on a corresponding set of stitched images in at least one set of stitched images, to obtain at least one throwing event detection result.
在本公开的实施例中,抛洒事件检测装置在获得至少一个待测车辆对应的至少一组拼接图像的情况下,即可利用抛洒事件检测模型,基于至少一组拼接图像对至少一个待测车辆进行抛洒事件检测,得到至少一个抛洒事件检测结果。In the embodiment of the present disclosure, the throwing event detection device can use the throwing event detection model to detect at least one vehicle to be tested based on the at least one set of stitched images when at least one set of stitched images corresponding to at least one vehicle to be tested is obtained. Detecting a spilling event is performed, and at least one spilling event detection result is obtained.
需要说明的是,在本公开的实施例中,抛洒事件检测装置利用抛洒事件检测模型,针对至少一个待测车辆中每个待测车辆,基于至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测之前,还可以执行以下步骤:获取车辆抛洒样本和预设时序差值神经网络;利用车辆抛洒样本对预设时序差值神经网络进行车辆抛洒事件检测训练,得到抛洒事件检测模型。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device utilizes the throwing event detection model, for each vehicle to be tested in at least one vehicle to be tested, based on a corresponding set of stitched images in at least one set of stitched images. Before the detection of the throwing event, the following steps may also be performed: obtaining a vehicle throwing sample and a preset time sequence difference neural network; using the vehicle throwing sample to train the vehicle throwing event detection on the preset time sequence difference neural network to obtain a throwing event detection model.
可以理解的是,在本公开的实施例中,抛洒事件检测装置获取车辆抛洒样本,用于进行预设时序差值神经网络的训练,从而可以得到精度较高的抛洒事件检测模型,保证抛洒事件检测的准确性。此外,抛洒事件检测模型本质上为一个已训练的时序差值神经网络,该网络可以从拼接图像中获得同一位置,不同时间的图像信息,从而可以准确进行抛洒事件检测,简单且高效。It can be understood that, in the embodiment of the present disclosure, the throwing event detection device obtains vehicle throwing samples, which are used for training the preset time sequence difference neural network, so that a throwing event detection model with higher accuracy can be obtained to ensure the throwing event. detection accuracy. In addition, the sprinkling event detection model is essentially a trained time-series difference neural network, which can obtain image information at the same location and at different times from the stitched images, so that the sprinkling event detection can be performed accurately, which is simple and efficient.
可以理解的是,在本公开的实施例中,抛洒事件检测模型可以是抛洒事件检测装置利用车辆抛洒样本对预设时序差值神经网络训练后得到的,具体的车辆抛洒样本和预设时序差值神经网络的结构可以根据实际需求设置和选择,本公开实施例不作限定。It can be understood that, in the embodiment of the present disclosure, the throwing event detection model may be obtained by the throwing event detection device using the vehicle throwing samples to train the preset time sequence difference neural network. The specific vehicle throwing samples and the preset timing difference are obtained. The structure of the value neural network can be set and selected according to actual requirements, which is not limited in the embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置在获取到一个待测车辆对应的一组拼接图像之后,可以将其输入抛洒事件检测模型,针对该组拼接图像组中的每个图像拼接的两部分进行差异比较,输出差异分数,即可根据差异分数确定出该待测车 辆是否存在抛洒事件,例如,在差异分数超过预设差异阈值时,确定该待测车辆存在抛洒事件,在差异分数小于预设差异阈值时,确定该待测车辆不存在抛洒事件。It should be noted that, in the embodiments of the present disclosure, after acquiring a set of stitched images corresponding to a vehicle to be tested, the throwing event detection device can input the stitched images into the throwing event detection model, and for each stitched image group in the set of stitched images Compare the difference between the two parts of the image stitching, output the difference score, and then determine whether the vehicle under test has a throwing event according to the difference score. For example, when the difference score exceeds a preset difference threshold, it is determined that the vehicle under test has a throwing event. , when the difference score is less than the preset difference threshold, it is determined that the vehicle to be tested does not have a throwing event.
需要说明的是,在本公开的实施例中,抛洒事件检测装置在执行步骤S103之前,即利用抛洒事件检测模型,基于至少一组拼接图像对至少一个待测车辆中每个待测车辆进行抛洒事件检测之前,还可以执行以下步骤:利用参考帧对当前帧进行疑似抛洒样本判断,得到样本判断结果。抛洒事件检测装置利用抛洒事件检测模型,基于至少一组拼接图像对至少一个待测车辆中每个待测车辆进行抛洒事件检测,可以包括:在样本判断结果为疑似抛洒样本的情况下,利用抛洒事件检测模型,基于至少一组拼接图像对至少一个待测车辆中每个待测车辆进行抛洒事件检测。It should be noted that, in the embodiment of the present disclosure, before performing step S103, the throwing event detection device uses the throwing event detection model to throw each vehicle under test in at least one vehicle under test based on at least one set of stitched images. Before the event detection, the following steps may also be performed: use the reference frame to judge the suspected throwing sample of the current frame, and obtain the sample judgment result. The throwing event detection device uses a throwing event detection model to detect throwing events for each vehicle to be tested in at least one vehicle to be tested based on at least one set of stitched images, which may include: when the sample judgment result is a suspected throwing sample, using throwing An event detection model for detecting a throwing event for each vehicle to be tested in the at least one vehicle to be tested based on at least one set of stitched images.
具体的,在本公开的实施例中,抛洒事件检测装置利用参考帧对当前帧进行疑似抛洒样本判断,得到样本判断结果,包括:将当前帧中每个像素点与参考帧中与每个像素点对应位置的像素点,在不同色彩通道上分别作差后取绝对值,并对得到的不同色彩通道的绝对差值取均值,得到第一差值矩阵;对第一差值矩阵进行低通滤波处理,得到第二差值矩阵;对第二差值矩阵取均值,得到目标评估均值;在目标评估均值大于预设均值阈值的情况下,确定样本判断结果为疑似抛洒样本;在目标评估均值不大于预设均值阈值的情况下,确定样本判断结果为非疑似抛洒样本。Specifically, in the embodiment of the present disclosure, the throwing event detection device uses the reference frame to judge the suspected throwing sample of the current frame, and obtains the sample judgment result, including: comparing each pixel in the current frame with each pixel in the reference frame and each pixel The pixel points at the corresponding position of the point, take the absolute value after making the difference on different color channels, and take the average value of the absolute difference value of the obtained different color channels to obtain the first difference value matrix; perform low-pass on the first difference value matrix. Filtering to obtain a second difference matrix; taking the average of the second difference matrix to obtain the target evaluation mean; in the case that the target evaluation mean is greater than the preset mean threshold, it is determined that the sample judgment result is a suspected throwing sample; in the target evaluation mean If it is not greater than the preset mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
需要说明的是,在本公开的实施例中,抛洒事件检测装置可以将当前帧中每个像素点,与参考帧中对应位置的像素点,在RGB通道上分别作差后取绝对值,并进一步在RGB通道取均值,从而得到第一差值矩阵。之后,抛洒事件检测装置在第一差值矩阵上进行低通滤波后取均值,得到目标评估均值。如果目标评估均值大于预设均值阈值,即样本判断结果为疑似抛洒样本,即表征当前帧与参考帧整体图像信息差异较大,因此,当前帧包括的待测车辆很可能存在抛洒事件,即可以执行步骤S103。如果目标评估均值不大于预设均值阈值,即样本判断结果为非疑似抛洒样本,则表征当前帧与参考帧整体图像信息基本一致,因此,当前帧包括的待测车辆大概率不存在抛洒事件,后续抛洒事件检测装置可以不对待测车辆进行抛洒事件检测,即可以不执行步骤S103,避免不必要的检测,从而提高检测效率,降低抛洒事件检测装置的功耗。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device can take the absolute value of each pixel in the current frame and the pixel at the corresponding position in the reference frame after making a difference on the RGB channel, and calculate the difference. Further take the mean value in the RGB channel to obtain the first difference matrix. Afterwards, the throwing event detection device performs low-pass filtering on the first difference matrix and takes an average value to obtain the target evaluation average value. If the target evaluation mean is greater than the preset mean threshold, that is, the sample judgment result is a suspected throwing sample, which means that the overall image information of the current frame and the reference frame is quite different. Therefore, the vehicle under test included in the current frame is likely to have a throwing event, that is, it can be Step S103 is performed. If the target evaluation mean is not greater than the preset mean threshold, that is, the sample judgment result is a non-suspected throwing sample, it means that the overall image information of the current frame and the reference frame is basically the same. Therefore, the vehicle under test included in the current frame has a high probability that there is no throwing event. The subsequent throwing event detection device may not perform throwing event detection on the vehicle to be tested, that is, step S103 may not be performed to avoid unnecessary detection, thereby improving detection efficiency and reducing power consumption of the throwing event detection device.
需要说明的是,在本公开的实施例中,抛洒事件检测装置利用参考帧对当前帧进行疑似抛洒样本检测,得到样本判断结果之前,还可以执行以下步骤:基于当前帧和参考帧进行拍摄镜头移动判断,得到移动判断结果。抛洒事件检测装置利用参考帧对当前帧进行疑似抛洒样本判断,得到样本判断结果,包括:在移动判断结果为未移动的情况下,利用参考帧对当前帧进行疑似抛洒样本判断,得到样本判断结果。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device uses the reference frame to detect the suspected throwing sample of the current frame, and before obtaining the sample judgment result, the following steps may also be performed: taking a shot based on the current frame and the reference frame. The movement judgment is obtained, and the movement judgment result is obtained. The throwing event detection device uses the reference frame to judge the suspected throwing sample of the current frame, and obtains the sample judgment result, including: when the movement judgment result is no movement, using the reference frame to judge the suspected throwing sample of the current frame, and obtain the sample judgment result .
具体的,在本公开的实施例中,抛洒事件检测装置基于当前帧和参考帧进行拍摄镜头移动判断,得到移动判断结果,包括:利用车辆检测模型对参考帧进行车辆检测,得到参考帧包括的车辆;将当前帧中每个像素点与参考帧中与每个像素点对应位置的像素点,在不同色彩通道上分别作差,并对得到的不同色彩通道的差值取均值,得到待处理差值矩阵;将待处理差值矩阵中,参考帧包括的车辆和当前帧中至少一个待测车辆对应的元素替换为零,并对替换后的差值矩阵取绝对值,得到已处理差值矩阵;计算已处理差值矩阵的平均值,得到背景差异值;在背景差异值大于预设差异阈值的情况下,确定移动判断结果为移动;在背景差异值不大于预设差异阈值的情况下,确定移动判断结果为未移动。Specifically, in the embodiment of the present disclosure, the throwing event detection device determines the movement of the shooting lens based on the current frame and the reference frame, and obtains the movement determination result. Vehicle; make a difference between each pixel point in the current frame and the pixel point corresponding to each pixel point in the reference frame on different color channels, and take the average of the obtained difference values of different color channels to obtain the pending processing Difference matrix; replace the elements corresponding to the vehicles included in the reference frame and at least one vehicle to be tested in the current frame in the difference matrix to be processed to zero, and take the absolute value of the replaced difference matrix to obtain the processed difference matrix; calculate the average value of the processed difference matrix to obtain the background difference value; in the case that the background difference value is greater than the preset difference threshold, it is determined that the movement judgment result is movement; in the case that the background difference value is not greater than the preset difference threshold , and determine that the movement judgment result is no movement.
需要说明的是,在本公开的实施例中,抛洒事件检测装置可以将当前帧中每个像素点,与参考帧中对应位置的像素点,在RGB通道上分别作差,并进一步在RGB通道取均值,从而得到待处理差值矩阵。由于判断拍摄镜头移动是基于图像背景的差异,因此,将待处理差值矩阵中两帧图像对应的元素替换为零,并进一步取绝对值,从而得到已处 理差值矩阵,最后进行已处理差值矩阵的均值计算,得到背景差异值。如果背景差异值大于预设差异阈值,即移动判断结果为移动,则表征拍摄镜头采集当前帧和参考帧时并非处于同一位置,拍摄镜头在采集当前帧和参考帧的过程中出现移动,参考帧与当前帧并非不同时刻同一位置的图像,因此,不能利用该参考帧进行后续抛洒事件检测。如果背景差异值不大于预设差异阈值,即移动判断结果为未移动,则表征拍摄镜头采集当前帧和参考帧时处于同一位置,拍摄镜头在采集当前帧和参考帧的过程中未移动,参考帧与当前帧为不同时刻同一位置的图像,因此,可以利用该参考帧进行后续抛洒事件检测,从而提高检测效率。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device may make a difference between each pixel in the current frame and the pixel at the corresponding position in the reference frame on the RGB channel, and further detect the difference between the RGB channel. Take the mean to get the difference matrix to be processed. Since the judgment of the movement of the shooting lens is based on the difference of the image background, the elements corresponding to the two frames of images in the difference matrix to be processed are replaced by zero, and the absolute value is further taken to obtain the processed difference matrix, and finally the processed difference matrix is processed. Calculate the mean of the value matrix to obtain the background difference value. If the background difference value is greater than the preset difference threshold, that is, the result of the movement judgment is movement, it means that the shooting lens is not in the same position when collecting the current frame and the reference frame, and the shooting lens moves during the process of collecting the current frame and the reference frame, and the reference frame It is not an image at the same position at a different time from the current frame, so the reference frame cannot be used for subsequent throwing event detection. If the background difference value is not greater than the preset difference threshold, that is, the movement judgment result is no movement, it means that the shooting lens is in the same position when collecting the current frame and the reference frame, and the shooting lens does not move during the process of collecting the current frame and the reference frame. The frame and the current frame are images of the same position at different times, so the reference frame can be used to detect subsequent throwing events, thereby improving the detection efficiency.
需要说明的是,在本公开的实施例中,抛洒事件检测装置中存储有预设差异阈值,用于衡量参考帧和当前帧的背景差异程度。具体的预设差异阈值可以根据实际需求和应用场景设定,本公开实施例不作限定。It should be noted that, in the embodiment of the present disclosure, a preset difference threshold is stored in the throwing event detection device, which is used to measure the background difference degree between the reference frame and the current frame. The specific preset difference threshold may be set according to actual requirements and application scenarios, which is not limited in this embodiment of the present disclosure.
需要说明的是,在本公开的实施例中,抛洒事件检测装置可以采用上述方式进行拍摄镜头移动判断,还可以采用其它方式进行判断,例如,从当前帧和参考帧中分别对应选取四个边角位置的图像区域进行差异比较。具体的拍摄镜头移动判断方式本公开实施例不作限定。It should be noted that, in the embodiment of the present disclosure, the throwing event detection device may use the above method to determine the movement of the shooting lens, or may also use other methods to determine, for example, correspondingly select four sides from the current frame and the reference frame. Differential comparison of image areas at corner positions. The specific manner of judging the movement of the shooting lens is not limited in the embodiment of the present disclosure.
本公开实施例提供了一种抛洒事件检测方法,包括:利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;获取当前帧对应的参考帧,并针对至少一个待测车辆中每个待测车辆,将当前帧和参考帧中对应同一位置的图像进行拼接,得到至少一个待测车辆对应的至少一组拼接图像;同一位置为当前帧和参考帧中待测车辆周围的位置;利用抛洒事件检测模型,针对至少一个待测车辆中每个待测车辆,基于至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。本公开的抛洒事件检测方法,采用车辆检测模型先从图像中进行车辆检测,再利用抛洒事件检测模型,基于车辆对应的拼接图像进行抛洒事件检测,不仅将抛洒事件与具体车辆关联起来,还提高了抛洒事件检测性能。An embodiment of the present disclosure provides a method for detecting a throwing event, including: using a vehicle detection model to perform vehicle detection on a current frame, and determining each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested; The reference frame corresponding to the frame, and for each vehicle to be tested in the at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are stitched to obtain at least one set of stitched images corresponding to at least one vehicle to be tested; the same The position is the position around the vehicle to be tested in the current frame and the reference frame; using the throwing event detection model, for each vehicle to be tested in at least one vehicle to be tested, the throwing event is performed based on a corresponding set of stitched images in at least one set of stitched images Detect, and obtain at least one throwing event detection result. The throwing event detection method of the present disclosure uses a vehicle detection model to first perform vehicle detection from an image, and then uses the throwing event detection model to detect the throwing event based on the stitched images corresponding to the vehicle, which not only associates the throwing event with a specific vehicle, but also improves the Improved spill event detection performance.
本公开实施例还提供了一种抛洒事件检测装置。图5为本公开实施例提供的一种抛洒事件检测装置的结构示意图。如图5所示,抛洒事件检测装置包括:The embodiment of the present disclosure also provides a throwing event detection device. FIG. 5 is a schematic structural diagram of a throwing event detection device according to an embodiment of the present disclosure. As shown in Figure 5, the throwing event detection device includes:
检测模块501,配置为利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;The detection module 501 is configured to use the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested, to obtain at least one vehicle to be tested;
拼接模块502,配置为获取所述当前帧对应的参考帧,并针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像;所述同一位置为所述当前帧和所述参考帧中所述待测车辆周围的位置;The splicing module 502 is configured to obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and the image corresponding to the same position in the reference frame to obtain at least one group of stitched images corresponding to the at least one vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
所述检测模块501,还配置为利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。The detection module 501 is further configured to use a throwing event detection model to detect a throwing event based on a corresponding set of stitched images in the at least one set of stitched images for each vehicle to be tested in the at least one vehicle to be tested, Obtain at least one spill event detection result.
在本公开一实施例中,所述拼接模块502,具体配置为获取采集到所述当前帧之前预设时长内采集到的至少一帧图像;利用所述车辆检测模型,对所述至少一帧图像中每一帧图像分别进行车辆检测,并统计所述每一帧图像检测出的车辆数量;将所述至少一帧图像中检测出的所述车辆数量最少的帧图像确定为所述参考帧。In an embodiment of the present disclosure, the splicing module 502 is specifically configured to acquire at least one frame of image collected within a preset time period before the current frame is collected; Vehicle detection is performed on each frame of image in the image, and the number of vehicles detected in each frame of image is counted; the frame image with the least number of vehicles detected in the at least one frame of image is determined as the reference frame .
在本公开一实施例中,所述检测模块501,还配置为利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果;In an embodiment of the present disclosure, the detection module 501 is further configured to use the reference frame to perform a sample judgment on the current frame suspected to be thrown, to obtain a sample judgment result;
所述检测模块501,具体配置为在所述样本判断结果为疑似抛洒样本的情况下,利用所述抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测。The detection module 501 is specifically configured to, when the sample judgment result is a suspected throwing sample, use the throwing event detection model to detect each of the at least one vehicle to be tested based on the at least one set of stitched images. The vehicle to be tested is tested for spillage events.
在本公开一实施例中,所述检测模块501,具体配置为将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差后取绝对值,并对得到的不同色彩通道的绝对差值取均值,得到第一差值矩阵;对所述第一差值矩阵进行低通滤波处理,得到第二差值矩阵;对所述第二差值矩阵取均值,得到目标评估均值;在所述目标评估均值大于预设均值阈值的情况下,确定所述样本判断结果为疑似抛洒样本;在所述目标评估均值不大于所述预设均值阈值的情况下,确定所述样本判断结果为非疑似抛洒样本。In an embodiment of the present disclosure, the detection module 501 is specifically configured to compare each pixel in the current frame and a pixel at a position corresponding to each pixel in the reference frame in different color channels Take the absolute value after making the difference respectively, and take the mean value of the absolute difference values of the obtained different color channels to obtain the first difference value matrix; perform low-pass filtering processing on the first difference value matrix to obtain the second difference value matrix Take the mean value of the second difference matrix to obtain the target evaluation mean value; in the case that the target evaluation mean value is greater than the preset mean value threshold, it is determined that the sample judgment result is a suspected throwing sample; when the target evaluation mean value is not When the value is greater than the preset mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
在本公开一实施例中,所述检测模块501,还配置为基于所述当前帧和所述参考帧进行拍摄镜头移动判断,得到移动判断结果;In an embodiment of the present disclosure, the detection module 501 is further configured to determine the movement of the shooting lens based on the current frame and the reference frame, and obtain a movement determination result;
所述检测模块501,具体配置为在所述移动判断结果为未移动的情况下,利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到所述样本判断结果。The detection module 501 is specifically configured to use the reference frame to perform a sample judgment on a suspected throwing sample of the current frame when the movement judgment result is no movement, to obtain the sample judgment result.
在本公开一实施例中,所述检测模块501,具体配置为利用所述车辆检测模型对所述参考帧进行车辆检测,得到所述参考帧包括的车辆;将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差,并对得到的不同色彩通道的差值取均值,得到待处理差值矩阵;将所述待处理差值矩阵中,所述参考帧包括的车辆和所述当前帧中所述至少一个待测车辆对应的元素替换为零,并对替换后的差值矩阵取绝对值,得到已处理差值矩阵;计算所述已处理差值矩阵的平均值,得到背景差异值;在所述背景差异值大于预设差异阈值的情况下,确定所述移动判断结果为移动;在所述背景差异值不大于所述预设差异阈值的情况下,确定所述移动判断结果为未移动。In an embodiment of the present disclosure, the detection module 501 is specifically configured to perform vehicle detection on the reference frame by using the vehicle detection model to obtain vehicles included in the reference frame; The point and the pixel point corresponding to each pixel point in the reference frame, respectively make differences on different color channels, and take the average value of the obtained difference values of the different color channels to obtain the difference value matrix to be processed; In the difference matrix to be processed, the elements corresponding to the vehicle included in the reference frame and the at least one vehicle to be tested in the current frame are replaced by zero, and the absolute value of the replaced difference matrix is taken to obtain the processing the difference matrix; calculating the average value of the processed difference matrix to obtain a background difference value; when the background difference value is greater than a preset difference threshold, determine that the movement judgment result is movement; in the background When the difference value is not greater than the preset difference threshold, it is determined that the movement judgment result is no movement.
在本公开一实施例中,所述拼接模块502,具体配置为在所述当前帧中,所述至少一个待测车辆中每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框;从所述参考帧中,分别选取与所述至少一组检测候选框位置一一对应的至少一组匹配候选框;将所述至少一组检测候选框与所述至少一组匹配候选框中,位置对应的候选框进行拼接,得到所述至少一组拼接图像。In an embodiment of the present disclosure, the splicing module 502 is specifically configured to, in the current frame, generate a set of detection candidate frames around each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one set of detection candidate frames candidate frame; from the reference frame, respectively select at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames; compare the at least one set of detection candidate frames with the at least one set of matching candidate frames frame, the candidate frames corresponding to the positions are spliced to obtain the at least one set of spliced images.
在本公开一实施例中,所述拼接模块502,具体配置为在所述当前帧中,第一车辆周围生成第一候选框;所述第一车辆为所述至少一个待测车辆中任一待测车辆;对所述第一候选框中至少一部分候选框进行缩减;从缩减后的候选框中选取出满足预设尺寸条件的候选框,与所述第一候选框中未缩减的候选框组成所述第一车辆对应的一组检测候选框。In an embodiment of the present disclosure, the splicing module 502 is specifically configured to generate a first candidate frame around a first vehicle in the current frame; the first vehicle is any one of the at least one vehicle to be tested vehicle to be tested; reduce at least a part of the candidate frames in the first candidate frame; select candidate frames that meet the preset size conditions from the reduced candidate frames, and candidate frames that are not reduced in the first candidate frame A group of detection candidate frames corresponding to the first vehicle is formed.
在上述装置中,所述拼接模块502,具体配置为对所述第一候选框中,位于所述第一车辆下方的候选框,按照预设缩减尺度进行缩减。In the above device, the splicing module 502 is specifically configured to reduce the candidate frame located under the first vehicle in the first candidate frame according to a preset reduction scale.
在上述装置中,所述拼接模块502,具体配置为缩减所述第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,所述第二车辆为所述至少一个待测车辆中与所述第一车辆不同的待测车辆。In the above device, the splicing module 502 is specifically configured to reduce the first candidate frame to the candidate frame that overlaps with the second candidate frame generated around the second vehicle; wherein the second vehicle is the at least one candidate frame. A vehicle to be tested that is different from the first vehicle in a vehicle to be tested.
在本公开一实施例中,抛洒事件检测装置还包括训练模块(图中未示出);In an embodiment of the present disclosure, the throwing event detection device further includes a training module (not shown in the figure);
所述训练模块,配置为获取车辆检测样本和预设检测模型;利用所述车辆检测样本对所述预设检测模型进行车辆检测训练,得到所述车辆检测模型;The training module is configured to obtain vehicle detection samples and a preset detection model; perform vehicle detection training on the preset detection model by using the vehicle detection samples to obtain the vehicle detection model;
以及,获取车辆抛洒样本和预设时序差值神经网络;利用所述车辆抛洒样本对所述预设时序差值神经网络进行车辆抛洒事件检测训练,得到所述抛洒事件检测模型。And, obtaining vehicle throwing samples and a preset timing difference neural network; using the vehicle throwing samples to perform vehicle throwing event detection training on the preset timing difference neural network to obtain the throwing event detection model.
本公开实施例还提供了一种电子设备。图6为本公开实施例提供的一种电子设备的结构示意图。如图6所示,电子设备包括:处理器601、存储器602和通信总线603;Embodiments of the present disclosure also provide an electronic device. FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 6 , the electronic device includes: a processor 601, a memory 602 and a communication bus 603;
所述通信总线603,配置为实现所述处理器601和所述存储器602之间的通信连接;The communication bus 603 is configured to realize the communication connection between the processor 601 and the memory 602;
所述处理器601,配置为执行所述存储器602中存储的一个或多个程序,以实现上 述抛洒事件检测方法。The processor 601 is configured to execute one or more programs stored in the memory 602, so as to implement the above-mentioned method for detecting a throwing event.
本公开实施例还提供了一种计算机可读存储介质,所述计算机可读存储介质存储有一个或者多个程序,所述一个或者多个程序可以被一个或者多个处理器执行,以实现上述抛洒事件检测方法。计算机可读存储介质可以是是易失性存储器(volatile memory),例如随机存取存储器(Random-Access Memory,RAM);或者非易失性存储器(non-volatile memory),例如只读存储器(Read-Only Memory,ROM),快闪存储器(flash memory),硬盘(Hard Disk Drive,HDD)或固态硬盘(Solid-State Drive,SSD);也可以是包括上述存储器之一或任意组合的各自设备,如移动电话、计算机、平板设备、个人数字助理等。Embodiments of the present disclosure further provide a computer-readable storage medium, where one or more programs are stored in the computer-readable storage medium, and the one or more programs can be executed by one or more processors to implement the above-mentioned Spill event detection method. The computer-readable storage medium may be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read Only Memory) -Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); it can also be a respective device including one or any combination of the above memories, Such as mobile phones, computers, tablet devices, personal digital assistants, etc.
本公开实施例还提供了一种计算机程序产品,所述计算机程序产品包括计算机程序或指令,在所述计算机程序或指令在计算机上运行的情况下,使所述计算机执行上述抛洒事件检测方法。An embodiment of the present disclosure also provides a computer program product, the computer program product includes a computer program or an instruction, and when the computer program or instruction runs on a computer, the computer is made to execute the above method for detecting a throwing event.
本领域内的技术人员应明白,本公开的实施例可提供为方法、系统、或计算机程序产品。因此,本公开可采用硬件实施例、软件实施例、或结合软件和硬件方面的实施例的形式。而且,本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器和光学存储器等)上实施的计算机程序产品的形式。As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.
本公开是参照根据本公开实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程信号处理设备的处理器以产生一个机器,使得通过计算机或其他可编程信号处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable signal processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable signal processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程信号处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable signal processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.
这些计算机程序指令也可装载到计算机或其他可编程信号处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded on a computer or other programmable signal processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.
以上所述,仅为本公开的较佳实施例而已,并非用于限定本公开的保护范围。The above descriptions are merely preferred embodiments of the present disclosure, and are not intended to limit the protection scope of the present disclosure.
工业实用性Industrial Applicability
本公开实施例提供了一种抛洒事件检测方法、装置、电子设备、存储介质及计算机程序产品,方法包括:利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;获取当前帧对应的参考帧,并针对至少一个待测车辆中每个待测车辆,将当前帧和参考帧中对应同一位置的图像进行拼接,得到至少一个待测车辆对应的至少一组拼接图像;同一位置为当前帧和参考帧中待测车辆周围的位置;利用抛洒事件检测模型,针对至少一个待测车辆中每个待测车辆,基于至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。本公开的抛洒事件检测方法,采用车辆检测模型先从图像中进行车辆检测,再利用抛洒事件检测模型,基于车辆对应的拼接图像进行抛洒事件检测,不仅将抛洒事件 与具体车辆关联起来,还提高了抛洒事件检测性能。Embodiments of the present disclosure provide a throwing event detection method, device, electronic device, storage medium and computer program product. The method includes: using a vehicle detection model to perform vehicle detection on a current frame, and determining each detected vehicle as a Vehicles to be tested, at least one vehicle to be tested is obtained; a reference frame corresponding to the current frame is obtained, and for each vehicle to be tested in at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are spliced to obtain at least one vehicle to be tested. At least one group of stitched images corresponding to a vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame; using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, based on at least one A set of spliced images corresponding to the set of spliced images is subjected to throwing event detection, and at least one throwing event detection result is obtained. The throwing event detection method of the present disclosure uses a vehicle detection model to first perform vehicle detection from an image, and then uses the throwing event detection model to detect the throwing event based on the stitched images corresponding to the vehicle, which not only associates the throwing event with a specific vehicle, but also improves the Improved spill event detection performance.

Claims (25)

  1. 一种抛洒事件检测方法,所述方法包括:A throwing event detection method, the method comprises:
    利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;Use the vehicle detection model to perform vehicle detection on the current frame, and determine each detected vehicle as a vehicle to be tested to obtain at least one vehicle to be tested;
    获取所述当前帧对应的参考帧,并针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像;所述同一位置为所述当前帧和所述参考帧中所述待测车辆周围的位置;Obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and an image corresponding to the same position in the reference frame to obtain the at least one vehicle to be tested at least one group of stitched images corresponding to the vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
    利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。Using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, throwing event detection is performed based on a corresponding set of stitched images in the at least one set of stitched images, to obtain at least one throwing event detection result.
  2. 根据权利要求1所述的方法,其中,所述获取所述当前帧对应的参考帧,包括:The method according to claim 1, wherein the obtaining the reference frame corresponding to the current frame comprises:
    获取采集到所述当前帧之前预设时长内采集到的至少一帧图像;acquiring at least one frame of image collected within a preset time period before the current frame is collected;
    利用所述车辆检测模型,对所述至少一帧图像中每一帧图像分别进行车辆检测,并统计所述每一帧图像检测出的车辆数量;Using the vehicle detection model, vehicle detection is performed on each frame of the at least one frame of image, and the number of vehicles detected in each frame of image is counted;
    将所述至少一帧图像中检测出的所述车辆数量最少的帧图像确定为所述参考帧。A frame image with the least number of vehicles detected in the at least one frame image is determined as the reference frame.
  3. 根据权利要求1所述的方法,其中,所述利用抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测之前,所述方法还包括:The method according to claim 1 , wherein, before the detection of a throwing event is performed on each vehicle under test in the at least one vehicle under test based on the at least one set of stitched images by using a throwing event detection model, the method Also includes:
    利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果;Use the reference frame to judge the suspected throwing sample of the current frame to obtain a sample judgment result;
    所述利用抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测,包括:The said using the throwing event detection model, based on the at least one group of stitched images, to perform throwing event detection on each vehicle to be tested in the at least one vehicle to be tested, including:
    在所述样本判断结果为疑似抛洒样本的情况下,利用所述抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测。In the case that the sample judgment result is a suspected throwing sample, the throwing event detection model is used to perform throwing event detection for each vehicle to be tested in the at least one vehicle to be tested based on the at least one set of stitched images.
  4. 根据权利要求3所述的方法,其中,所述利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果,包括:The method according to claim 3, wherein the use of the reference frame to perform a suspected throwing sample judgment on the current frame to obtain a sample judgment result, comprising:
    将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差后取绝对值,并对得到的不同色彩通道的绝对差值取均值,得到第一差值矩阵;Each pixel in the current frame and the pixel at the position corresponding to each pixel in the reference frame are compared on different color channels to obtain the absolute value, and the obtained values of the different color channels are compared. The absolute difference is averaged to obtain the first difference matrix;
    对所述第一差值矩阵进行低通滤波处理,得到第二差值矩阵;performing low-pass filtering processing on the first difference matrix to obtain a second difference matrix;
    对所述第二差值矩阵取均值,得到目标评估均值;Taking the mean value of the second difference matrix to obtain the mean value of the target evaluation;
    在所述目标评估均值大于预设均值阈值的情况下,确定所述样本判断结果为疑似抛洒样本;In the case that the target evaluation mean value is greater than the preset mean value threshold, it is determined that the sample judgment result is a suspected throwing sample;
    在所述目标评估均值不大于所述预设均值阈值的情况下,确定所述样本判断结果为非疑似抛洒样本。In the case that the target evaluation mean is not greater than the preset mean threshold, it is determined that the sample judgment result is a non-suspected throwing sample.
  5. 根据权利要求3所述的方法,其中,所述利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果之前,所述方法还包括;The method according to claim 3, wherein, before the sample judgment is performed on the current frame by using the reference frame, and the sample judgment result is obtained, the method further comprises:
    基于所述当前帧和所述参考帧进行拍摄镜头移动判断,得到移动判断结果;Based on the current frame and the reference frame, the movement judgment of the shooting lens is performed to obtain a movement judgment result;
    所述利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果,包括:The said reference frame is used to judge the suspected throwing sample of the current frame, and the sample judgment result is obtained, including:
    在所述移动判断结果为未移动的情况下,利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到所述样本判断结果。In the case that the movement judgment result is no movement, the sample judgment result of the sample is obtained by using the reference frame to judge the suspected throwing sample of the current frame.
  6. 根据权利要求5所述的方法,其中,所述基于所述当前帧和所述参考帧进行拍 摄镜头移动判断,得到移动判断结果,包括:method according to claim 5, wherein, described based on the current frame and the reference frame to carry out the camera movement judgment, obtain the movement judgment result, including:
    利用所述车辆检测模型对所述参考帧进行车辆检测,得到所述参考帧包括的车辆;Perform vehicle detection on the reference frame by using the vehicle detection model to obtain vehicles included in the reference frame;
    将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差,并对得到的不同色彩通道的差值取均值,得到待处理差值矩阵;Differentiate each pixel in the current frame and the pixel at the corresponding position of each pixel in the reference frame on different color channels, and average the obtained differences in different color channels. , get the difference matrix to be processed;
    将所述待处理差值矩阵中,所述参考帧包括的车辆和所述当前帧中所述至少一个待测车辆对应的元素替换为零,并对替换后的差值矩阵取绝对值,得到已处理差值矩阵;In the difference matrix to be processed, the vehicle included in the reference frame and the element corresponding to the at least one vehicle to be tested in the current frame are replaced by zero, and the absolute value of the replaced difference matrix is obtained to obtain Processed difference matrix;
    计算所述已处理差值矩阵的平均值,得到背景差异值;Calculate the average value of the processed difference matrix to obtain the background difference value;
    在所述背景差异值大于预设差异阈值的情况下,确定所述移动判断结果为移动;In the case that the background difference value is greater than a preset difference threshold, determine that the movement judgment result is movement;
    在所述背景差异值不大于所述预设差异阈值的情况下,确定所述移动判断结果为未移动。In the case that the background difference value is not greater than the preset difference threshold, it is determined that the movement determination result is no movement.
  7. 根据权利要求1所述的方法,其中,所述针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像,包括:The method according to claim 1, wherein, for each vehicle to be tested in the at least one vehicle to be tested, the images corresponding to the same position in the current frame and the reference frame are spliced to obtain the At least one group of stitched images corresponding to at least one vehicle to be tested, including:
    在所述当前帧中,所述至少一个待测车辆的每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框;In the current frame, a set of detection candidate frames is generated around each vehicle to be tested of the at least one vehicle to be tested, to obtain at least one set of detection candidate frames;
    从所述参考帧中,分别选取与所述至少一组检测候选框位置一一对应的至少一组匹配候选框;From the reference frame, respectively select at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames;
    将所述至少一组检测候选框与所述至少一组匹配候选框中,位置对应的候选框进行拼接,得到所述至少一组拼接图像。The at least one set of detection candidate frames and the candidate frames corresponding to the positions of the at least one set of matching candidate frames are spliced to obtain the at least one set of spliced images.
  8. 根据权利要求7所述的方法,其中,所述在所述当前帧中,所述至少一个待测车辆的每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框,包括:The method according to claim 7, wherein, in the current frame, a set of detection candidate frames is generated around each vehicle to be tested of the at least one vehicle to be tested, to obtain at least one set of detection candidate frames, comprising: :
    在所述当前帧中,第一车辆周围生成第一候选框;所述第一车辆为所述至少一个待测车辆中任一待测车辆;In the current frame, a first candidate frame is generated around the first vehicle; the first vehicle is any vehicle to be tested among the at least one vehicle to be tested;
    对所述第一候选框中至少一部分候选框进行缩减;reducing at least a part of the candidate frames in the first candidate frame;
    从缩减后的候选框中选取出满足预设尺寸条件的候选框,与所述第一候选框中未缩减的候选框组成所述第一车辆对应的一组检测候选框。A candidate frame that satisfies a preset size condition is selected from the reduced candidate frame, and the candidate frame that is not reduced in the first candidate frame forms a group of detection candidate frames corresponding to the first vehicle.
  9. 根据权利要求8所述的方法,其中,所述对所述第一候选框中至少一部分候选框进行缩减,包括:The method according to claim 8, wherein the reducing at least a part of the candidate frames in the first candidate frame comprises:
    对所述第一候选框中,位于所述第一车辆下方的候选框,按照预设缩减尺度进行缩减。For the first candidate frame, the candidate frame located below the first vehicle is reduced according to a preset reduction scale.
  10. 根据权利要求8所述的方法,其中,所述对所述第一候选框中至少一部分候选框进行缩减,包括:The method according to claim 8, wherein the reducing at least a part of the candidate frames in the first candidate frame comprises:
    缩减所述第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,所述第二车辆为所述至少一个待测车辆中与所述第一车辆不同的待测车辆。Reduce the first candidate frame to a candidate frame that overlaps with the second candidate frame generated around the second vehicle; wherein the second vehicle is the at least one vehicle to be tested that is different from the first vehicle. test vehicle.
  11. 根据权利要求1所述的方法,其中,所述利用车辆检测模型对当前帧进行车辆检测之前,所述方法还包括:The method according to claim 1, wherein before the vehicle detection is performed on the current frame by using the vehicle detection model, the method further comprises:
    获取车辆检测样本和预设检测模型;Obtain vehicle detection samples and preset detection models;
    利用所述车辆检测样本对所述预设检测模型进行车辆检测训练,得到所述车辆检测模型;Use the vehicle detection samples to perform vehicle detection training on the preset detection model to obtain the vehicle detection model;
    所述利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测之前,所述方法还包括:Before using the throwing event detection model, for each vehicle to be tested in the at least one vehicle to be tested, the method further includes:
    获取车辆抛洒样本和预设时序差值神经网络;Obtain vehicle throwing samples and preset time series difference neural network;
    利用所述车辆抛洒样本对所述预设时序差值神经网络进行车辆抛洒事件检测训练, 得到所述抛洒事件检测模型。Using the vehicle throwing samples to perform vehicle throwing event detection training on the preset time sequence difference neural network to obtain the throwing event detection model.
  12. 一种抛洒事件检测装置,包括:A throwing event detection device, comprising:
    检测模块,配置为利用车辆检测模型对当前帧进行车辆检测,并将检测出的每一个车辆确定为一个待测车辆,得到至少一个待测车辆;a detection module, configured to perform vehicle detection on the current frame by using a vehicle detection model, and determine each detected vehicle as a vehicle to be tested, to obtain at least one vehicle to be tested;
    拼接模块,配置为获取所述当前帧对应的参考帧,并针对所述至少一个待测车辆中每个待测车辆,将所述当前帧和所述参考帧中对应同一位置的图像进行拼接,得到所述至少一个待测车辆对应的至少一组拼接图像;所述同一位置为所述当前帧和所述参考帧中所述待测车辆周围的位置;a splicing module, configured to obtain a reference frame corresponding to the current frame, and for each vehicle to be tested in the at least one vehicle to be tested, splicing the current frame and an image corresponding to the same position in the reference frame, obtaining at least one group of stitched images corresponding to the at least one vehicle to be tested; the same position is the position around the vehicle to be tested in the current frame and the reference frame;
    所述检测模块,还配置为利用抛洒事件检测模型,针对所述至少一个待测车辆中每个待测车辆,基于所述至少一组拼接图像中对应的一组拼接图像进行抛洒事件检测,得到至少一个抛洒事件检测结果。The detection module is further configured to use a throwing event detection model to detect a throwing event based on a corresponding set of stitched images in the at least one set of stitched images for each vehicle to be tested in the at least one vehicle to be tested, and obtain: At least one spill event detection result.
  13. 根据权利要求12所述的装置,其中,The apparatus of claim 12, wherein,
    所述拼接模块,具体配置为获取采集到所述当前帧之前预设时长内采集到的至少一帧图像;利用所述车辆检测模型,对所述至少一帧图像中每一帧图像分别进行车辆检测,并统计所述每一帧图像检测出的车辆数量;将所述至少一帧图像中检测出的所述车辆数量最少的帧图像确定为所述参考帧。The splicing module is specifically configured to acquire at least one frame of image collected within a preset time period before the current frame is collected; and use the vehicle detection model to perform vehicle detection on each frame of the at least one frame of image. Detecting, and counting the number of vehicles detected in each frame of image; determining the frame image with the least number of vehicles detected in the at least one frame of image as the reference frame.
  14. 根据权利要求12所述的装置,其中,The apparatus of claim 12, wherein,
    所述检测模块,还配置为利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到样本判断结果;The detection module is further configured to use the reference frame to perform a sample judgment on the current frame suspected to be thrown, and obtain a sample judgment result;
    所述检测模块,具体配置为在所述样本判断结果为疑似抛洒样本的情况下,利用所述抛洒事件检测模型,基于所述至少一组拼接图像对所述至少一个待测车辆中每个待测车辆进行抛洒事件检测。The detection module is specifically configured to, when the sample judgment result is a suspected throwing sample, use the throwing event detection model, based on the at least one set of stitched images, to detect each of the at least one vehicle to be tested. Test vehicle for spill event detection.
  15. 根据权利要求14所述的装置,其中,The apparatus of claim 14, wherein,
    所述检测模块,具体配置为将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差后取绝对值,并对得到的不同色彩通道的绝对差值取均值,得到第一差值矩阵;对所述第一差值矩阵进行低通滤波处理,得到第二差值矩阵;对所述第二差值矩阵取均值,得到目标评估均值;在所述目标评估均值大于预设均值阈值的情况下,确定所述样本判断结果为疑似抛洒样本;在所述目标评估均值不大于所述预设均值阈值的情况下,确定所述样本判断结果为非疑似抛洒样本。The detection module is specifically configured to take the absolute value of each pixel in the current frame and the pixel at the position corresponding to each pixel in the reference frame, respectively, after making a difference on different color channels, and averaging the absolute difference values of the obtained different color channels to obtain a first difference value matrix; performing low-pass filtering on the first difference value matrix to obtain a second difference value matrix; for the second difference value matrix Take the average value to obtain the target evaluation average value; in the case that the target evaluation average value is greater than the preset average value threshold, determine that the sample judgment result is a suspected throwing sample; in the case where the target evaluation average value is not greater than the preset average value threshold value Next, it is determined that the sample judgment result is a non-suspected throwing sample.
  16. 根据权利要求14所述的装置,其中,The apparatus of claim 14, wherein,
    所述检测模块,还配置为基于所述当前帧和所述参考帧进行拍摄镜头移动判断,得到移动判断结果;The detection module is further configured to judge the movement of the shooting lens based on the current frame and the reference frame, and obtain a movement judgment result;
    所述检测模块,具体配置为在所述移动判断结果为未移动的情况下,利用所述参考帧对所述当前帧进行疑似抛洒样本判断,得到所述样本判断结果。The detection module is specifically configured to use the reference frame to judge a suspected throwing sample of the current frame when the movement judgment result is no movement, to obtain the sample judgment result.
  17. 根据权利要求16所述的装置,其中,The apparatus of claim 16, wherein,
    所述检测模块,具体配置为利用所述车辆检测模型对所述参考帧进行车辆检测,得到所述参考帧包括的车辆;将所述当前帧中每个像素点与所述参考帧中与所述每个像素点对应位置的像素点,在不同色彩通道上分别作差,并对得到的不同色彩通道的差值取均值,得到待处理差值矩阵;将所述待处理差值矩阵中,所述参考帧包括的车辆和所述当前帧中所述至少一个待测车辆对应的元素替换为零,并对替换后的差值矩阵取绝对值,得到已处理差值矩阵;计算所述已处理差值矩阵的平均值,得到背景差异值;在所述背景差异值大于预设差异阈值的情况下,确定所述移动判断结果为移动;在所述背景差异值不大于所述预设差异阈值的情况下,确定所述移动判断结果为未移动。The detection module is specifically configured to perform vehicle detection on the reference frame by using the vehicle detection model to obtain the vehicles included in the reference frame; The pixel points at the corresponding positions of each pixel point are respectively different in different color channels, and the obtained difference values of different color channels are averaged to obtain the difference value matrix to be processed; in the difference value matrix to be processed, The element corresponding to the vehicle included in the reference frame and the at least one vehicle to be tested in the current frame is replaced by zero, and the absolute value of the replaced difference matrix is taken to obtain a processed difference matrix; The average value of the difference matrix is processed to obtain a background difference value; in the case that the background difference value is greater than a preset difference threshold, it is determined that the movement judgment result is movement; when the background difference value is not greater than the preset difference In the case of the threshold value, it is determined that the movement determination result is no movement.
  18. 根据权利要求12所述的装置,其中,The apparatus of claim 12, wherein,
    所述拼接模块,具体配置为在所述当前帧中,所述至少一个待测车辆中每个待测车辆周围生成一组检测候选框,得到至少一组检测候选框;从所述参考帧中,分别选取与所述至少一组检测候选框位置一一对应的至少一组匹配候选框;将所述至少一组检测候选框与所述至少一组匹配候选框中,位置对应的候选框进行拼接,得到所述至少一组拼接图像。The splicing module is specifically configured to, in the current frame, generate a set of detection candidate frames around each vehicle to be tested in the at least one vehicle to be tested, to obtain at least one set of detection candidate frames; from the reference frame , respectively select at least one set of matching candidate frames corresponding to the positions of the at least one set of detection candidate frames; compare the at least one set of detection candidate frames with the candidate frames corresponding to the positions of the at least one set of matching candidate frames. Stitching to obtain the at least one set of stitched images.
  19. 根据权利要求18所述的装置,其中,The apparatus of claim 18, wherein,
    所述拼接模块,具体配置为在所述当前帧中,第一车辆周围生成第一候选框;所述第一车辆为所述至少一个待测车辆中任一待测车辆;对所述第一候选框中至少一部分候选框进行缩减;从缩减后的候选框中选取出满足预设尺寸条件的候选框,与所述第一候选框中未缩减的候选框组成所述第一车辆对应的一组检测候选框。The splicing module is specifically configured to generate a first candidate frame around the first vehicle in the current frame; the first vehicle is any vehicle to be tested in the at least one vehicle to be tested; At least a part of the candidate frames in the candidate frame is reduced; candidate frames that meet the preset size conditions are selected from the reduced candidate frames, and the candidate frames that are not reduced in the first candidate frame form a corresponding one of the first vehicle. Group detection candidate boxes.
  20. 根据权利要求19所述的装置,其中,The apparatus of claim 19, wherein,
    所述拼接模块,具体配置为对所述第一候选框中,位于所述第一车辆下方的候选框,按照预设缩减尺度进行缩减。The splicing module is specifically configured to reduce the candidate frame located under the first vehicle in the first candidate frame according to a preset reduction scale.
  21. 根据权利要求19所述的装置,其中,The apparatus of claim 19, wherein,
    所述拼接模块,具体配置为缩减所述第一候选框中,与第二车辆周围生成的第二候选框重叠的候选框;其中,所述第二车辆为所述至少一个待测车辆中与所述第一车辆不同的待测车辆。The splicing module is specifically configured to reduce the first candidate frame to a candidate frame that overlaps with a second candidate frame generated around a second vehicle; wherein the second vehicle is one of the at least one vehicle to be tested that is the same as the second candidate frame. The first vehicle is different from the vehicle to be tested.
  22. 根据权利要求12所述的装置,其中,还包括训练模块;The apparatus of claim 12, further comprising a training module;
    所述训练模块,配置为获取车辆检测样本和预设检测模型;利用所述车辆检测样本对所述预设检测模型进行车辆检测训练,得到所述车辆检测模型;The training module is configured to obtain vehicle detection samples and a preset detection model; perform vehicle detection training on the preset detection model by using the vehicle detection samples to obtain the vehicle detection model;
    以及,获取车辆抛洒样本和预设时序差值神经网络;利用所述车辆抛洒样本对所述预设时序差值神经网络进行车辆抛洒事件检测训练,得到所述抛洒事件检测模型。And, obtaining vehicle throwing samples and a preset timing difference neural network; using the vehicle throwing samples to perform vehicle throwing event detection training on the preset timing difference neural network to obtain the throwing event detection model.
  23. 一种电子设备,包括:处理器、存储器和通信总线;An electronic device, comprising: a processor, a memory and a communication bus;
    所述通信总线,配置为实现所述处理器和所述存储器之间的通信连接;the communication bus configured to implement a communication connection between the processor and the memory;
    所述处理器,配置为执行所述存储器中存储的一个或多个程序,以实现权利要求1-11任一项所述的抛洒事件检测方法。The processor is configured to execute one or more programs stored in the memory, so as to implement the throwing event detection method according to any one of claims 1-11.
  24. 一种计算机可读存储介质,所述计算机可读存储介质存储有一个或者多个程序,所述一个或者多个程序可以被一个或者多个处理器执行,以实现权利要求1-11任一项所述的抛洒事件检测方法。A computer-readable storage medium storing one or more programs that can be executed by one or more processors to realize any one of claims 1-11 The described throwing event detection method.
  25. 一种计算机程序产品,所述计算机程序产品包括计算机程序或指令,在所述计算机程序或指令在计算机上运行的情况下,使所述计算机执行权利要求1-11中任一项所述的抛洒事件检测方法。A computer program product, the computer program product comprising a computer program or instructions that, when the computer program or instructions are run on a computer, cause the computer to perform the throwing described in any one of claims 1-11 Event detection method.
PCT/CN2021/133487 2021-04-30 2021-11-26 Spill-out event detection method and apparatus, electronic device, storage medium, and computer program product WO2022227548A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110483379.6A CN113191270B (en) 2021-04-30 2021-04-30 Method and device for detecting throwing event, electronic equipment and storage medium
CN202110483379.6 2021-04-30

Publications (1)

Publication Number Publication Date
WO2022227548A1 true WO2022227548A1 (en) 2022-11-03

Family

ID=76983353

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/133487 WO2022227548A1 (en) 2021-04-30 2021-11-26 Spill-out event detection method and apparatus, electronic device, storage medium, and computer program product

Country Status (2)

Country Link
CN (1) CN113191270B (en)
WO (1) WO2022227548A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115546704A (en) * 2022-11-28 2022-12-30 城云科技(中国)有限公司 Vehicle projectile identification method, device and application
CN118942020A (en) * 2024-10-12 2024-11-12 浙江大华技术股份有限公司 A method, device and electronic equipment for determining spillage

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113191270B (en) * 2021-04-30 2022-11-11 北京市商汤科技开发有限公司 Method and device for detecting throwing event, electronic equipment and storage medium
CN113869275A (en) * 2021-10-15 2021-12-31 浙大宁波理工学院 A vehicle spill detection system based on mobile edge computing

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160093214A1 (en) * 2014-09-30 2016-03-31 Xerox Corporation Vision-based on-street parked vehicle detection via normalized-view classifiers and temporal filtering
CN109886219A (en) * 2019-02-26 2019-06-14 中兴飞流信息科技有限公司 Shed object detecting method, device and computer readable storage medium
CN110889371A (en) * 2019-11-26 2020-03-17 浙江大华技术股份有限公司 Method and device for detecting throwing of muck truck
CN111274982A (en) * 2020-02-04 2020-06-12 浙江大华技术股份有限公司 Method and device for identifying projectile and storage medium
CN111797727A (en) * 2020-06-18 2020-10-20 浙江大华技术股份有限公司 Method and device for detecting road surface sprinkled object and storage medium
CN113191270A (en) * 2021-04-30 2021-07-30 北京市商汤科技开发有限公司 Method and device for detecting throwing event, electronic equipment and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106297278B (en) * 2015-05-18 2019-12-20 杭州海康威视数字技术股份有限公司 Method and system for querying a projectile vehicle
US10254763B2 (en) * 2016-12-29 2019-04-09 Intel Corporation Detection of traffic dynamics and road changes in autonomous driving
CN112560546B (en) * 2019-09-10 2024-06-18 杭州海康威视数字技术股份有限公司 Method and device for detecting throwing behavior and storage medium
CN111753612B (en) * 2019-09-11 2024-06-14 上海高德威智能交通系统有限公司 A method, device and storage medium for detecting spilled objects
CN110991466A (en) * 2019-11-21 2020-04-10 同济大学 Highway road surface condition detecting system based on novel vision sensing equipment
CN111507237A (en) * 2020-04-13 2020-08-07 北京巨视科技有限公司 Video-based abnormal event detection method and device applied to expressway

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160093214A1 (en) * 2014-09-30 2016-03-31 Xerox Corporation Vision-based on-street parked vehicle detection via normalized-view classifiers and temporal filtering
CN109886219A (en) * 2019-02-26 2019-06-14 中兴飞流信息科技有限公司 Shed object detecting method, device and computer readable storage medium
CN110889371A (en) * 2019-11-26 2020-03-17 浙江大华技术股份有限公司 Method and device for detecting throwing of muck truck
CN111274982A (en) * 2020-02-04 2020-06-12 浙江大华技术股份有限公司 Method and device for identifying projectile and storage medium
CN111797727A (en) * 2020-06-18 2020-10-20 浙江大华技术股份有限公司 Method and device for detecting road surface sprinkled object and storage medium
CN113191270A (en) * 2021-04-30 2021-07-30 北京市商汤科技开发有限公司 Method and device for detecting throwing event, electronic equipment and storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115546704A (en) * 2022-11-28 2022-12-30 城云科技(中国)有限公司 Vehicle projectile identification method, device and application
CN115546704B (en) * 2022-11-28 2023-02-17 城云科技(中国)有限公司 Vehicle projectile identification method, device and application
CN118942020A (en) * 2024-10-12 2024-11-12 浙江大华技术股份有限公司 A method, device and electronic equipment for determining spillage
CN118942020B (en) * 2024-10-12 2025-02-14 浙江大华技术股份有限公司 A method, device and electronic equipment for determining spillage

Also Published As

Publication number Publication date
CN113191270B (en) 2022-11-11
CN113191270A (en) 2021-07-30

Similar Documents

Publication Publication Date Title
WO2022227548A1 (en) Spill-out event detection method and apparatus, electronic device, storage medium, and computer program product
US20200160040A1 (en) Three-dimensional living-body face detection method, face authentication recognition method, and apparatuses
CN112330601A (en) Parking detection method, device, equipment and medium based on fisheye camera
CN109670383B (en) Video shielding area selection method and device, electronic equipment and system
CN112336342B (en) Hand key point detection method and device and terminal equipment
CN109086734B (en) Method and device for positioning pupil image in human eye image
WO2016127736A1 (en) Computing method for area of fingerprint overlapping area and electronic apparatus
JP2020518078A (en) METHOD AND APPARATUS FOR OBTAINING VEHICLE LOSS EVALUATION IMAGE, SERVER, AND TERMINAL DEVICE
US20190156499A1 (en) Detection of humans in images using depth information
CN113688820B (en) Method, device and electronic device for identifying stroboscopic stripe information
US11126875B2 (en) Method and device of multi-focal sensing of an obstacle and non-volatile computer-readable storage medium
JP2014057303A (en) System and method for utilizing enhanced scene detection in depth estimation procedure
CN114842439B (en) Vehicle identification method, device, electronic device and storage medium across sensing devices
CN111553302B (en) Key frame selection method, apparatus, device, and computer-readable storage medium
CN109102026B (en) Vehicle image detection method, device and system
CN112926461B (en) Neural network training, driving control method and device
CN112784675B (en) Target detection method and device, storage medium and terminal
CN117635590B (en) Defect detection method and device for notebook computer shell
CN114267041A (en) Method and device for identifying object in scene
CN103763458A (en) Scene change detection method and device
CN117132768A (en) License plate and face detection and desensitization method and device, electronic equipment and storage medium
WO2018058573A1 (en) Object detection method, object detection apparatus and electronic device
CN110688926A (en) Subject detection method and apparatus, electronic device, and computer-readable storage medium
CN115690747B (en) Vehicle blind area detection model test method and device, electronic equipment and storage medium
CN107145892A (en) A Salient Object Detection Method in Image Based on Adaptive Fusion Mechanism

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21939007

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21939007

Country of ref document: EP

Kind code of ref document: A1