WO2022134387A1

WO2022134387A1 - Vehicle wrong-way travel detection method, apparatus, device, computer-readable storage medium, and computer program product

Info

Publication number: WO2022134387A1
Application number: PCT/CN2021/086694
Authority: WO
Inventors: 谈正; 朱铖恺; 薛志强; 路少卿; 武伟
Original assignee: 深圳市商汤科技有限公司
Priority date: 2020-12-21
Filing date: 2021-04-12
Publication date: 2022-06-30
Also published as: CN112750317A

Abstract

Provided are a vehicle wrong-way travel detection method, apparatus, device, computer-readable storage medium, and computer program product; the method comprises: determining trajectory information of a target vehicle in a video stream (S101); obtaining lane information in the video stream by means of a preset lane segmentation model, lane information comprising a corresponding vehicle travel area and vehicle travel direction for each lane among at least one lane (S102); according to the trajectory information and the vehicle travel area and vehicle travel direction of each lane, determining whether there is wrong-way travel behavior of the target vehicle (S103).

Description

Vehicle retrograde detection method, apparatus, device, computer-readable storage medium, and computer program product

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure is based on the Chinese patent application with the application number of 202011520348.5, the application date of December 21, 2020, and the application name of "vehicle retrograde detection method, device, equipment and computer-readable storage medium", and requires the Chinese patent application Priority, the entire content of this Chinese patent application is hereby incorporated by reference into the present disclosure.

technical field

The present disclosure relates to the technical field of vehicles, and in particular, to a vehicle retrograde detection method, apparatus, device, computer-readable storage medium, and computer program product.

Background technique

The wrong-way of vehicles is a very serious traffic violation. The wrong-way of a small number of vehicles may cause serious traffic accidents, seriously endangering road safety and road traffic efficiency. In order to achieve accurate vehicle retrograde detection, it is often through manual viewing to find out whether retrograde behavior occurs in surveillance videos. The real-time detection of surveillance video by manual is not only time-consuming and labor-intensive, but also has low detection efficiency, and is prone to delays and omissions.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide a vehicle retrograde detection method, apparatus, device, computer-readable storage medium, and computer program product.

The technical solutions of the embodiments of the present disclosure are implemented as follows:

An embodiment of the present disclosure provides a method for detecting the wrong direction of a vehicle, including: determining trajectory information of a target vehicle in a video stream; acquiring lane information in a video stream through a preset lane segmentation model, where the lane information includes each of the lanes in at least one lane. The driving area and driving direction corresponding to the lane; determine whether the target vehicle has retrograde behavior according to the trajectory information and the driving area and driving direction of each lane.

In an embodiment of the present disclosure, the determining whether the target vehicle has a retrograde behavior according to the trajectory information and the driving area and driving direction of each lane includes: determining, according to the position information of the target vehicle at multiple times in the trajectory information, The displacement direction of the target vehicle; according to the position information of the target vehicle at multiple times and the driving area of each lane, determine the target lane where the target vehicle is located; based on whether there is a deviation between the driving direction of the target lane and the displacement direction of the target vehicle, Determine if the target vehicle has retrograde behavior.

In an embodiment of the present disclosure, the obtaining the lane information in the video stream through the preset lane segmentation model includes: in response to the lane information obtaining condition, obtaining the lane information in the video stream through the lane segmentation model; obtaining the lane information The conditions include at least one of the following: when the image capture device rotates; when the image capture device is initialized; wherein, the image capture device is used to acquire image information in the current real scene and output a video stream.

In an embodiment of the present disclosure, a method for detecting rotation of an image capture device includes: extracting multiple image frames from a video stream; the multiple image frames at least include a current image frame and a historical image frame adjacent to the current image frame; Determine whether the image acquisition device rotates through the current image frame and the historical image frame.

In an embodiment of the present disclosure, the obtaining lane information in the video stream by using a preset lane segmentation model includes: inputting an image frame in the video stream into the lane segmentation model to obtain a segmentation result of the image frame, The segmentation result includes the driving area corresponding to each lane in at least one lane; according to the sample trajectory acquisition rule, the sample trajectories corresponding to multiple sample vehicles in the driving area corresponding to each lane are obtained; according to the driving area corresponding to each lane and each The sample trajectories corresponding to multiple sample vehicles in the driving area corresponding to the lane are used to determine the driving direction corresponding to each lane.

In an embodiment of the present disclosure, the sample trajectory includes a start position and an end position of the sample vehicle; track, and determine the driving direction corresponding to each lane, including: according to the starting positions and ending positions corresponding to a plurality of sample vehicles in the driving area corresponding to each lane, determining the driving area corresponding to each lane corresponding to a plurality of sample vehicles Track direction: according to the track directions corresponding to multiple sample vehicles in the driving area corresponding to each lane, determine the driving direction corresponding to each lane in at least one lane.

In an embodiment of the present disclosure, the sample trajectory acquisition rule includes at least one of the following: acquiring sample trajectories at preset time intervals; counting the number of acquired sample trajectories, when the number of sample trajectories reaches a preset number , stop the acquisition of sample trajectories.

In an embodiment of the present disclosure, the determining the driving direction corresponding to each lane in the at least one lane according to the trajectory directions corresponding to the plurality of sample vehicles in the driving area corresponding to each lane includes: determining the driving direction corresponding to each of the plurality of trajectory directions Normalize each trajectory direction of , to obtain a trajectory vector corresponding to each trajectory direction; determine the vector sum of multiple trajectory vectors corresponding to each lane as the direction corresponding to each lane.

In an embodiment of the present disclosure, the method further includes: determining a trajectory offset quantization value corresponding to each lane according to the driving direction corresponding to each lane and a plurality of trajectory vectors corresponding to each lane; wherein, the trajectory The offset quantization value is used to characterize the degree of dispersion between the multiple trajectory vectors corresponding to the lane; when the trajectory offset quantization value exceeds the preset precision threshold, the lane information is re-determined according to the video stream.

In an embodiment of the present disclosure, the determining whether the target vehicle has a retrograde behavior based on whether there is a deviation between the driving direction of the target lane and the displacement direction of the target vehicle includes: according to at least one sub-displacement direction corresponding to the displacement direction of the target vehicle At least one included angle is obtained from the included angle with the driving direction of the target lane; wherein, the displacement direction includes at least one sub-displacement direction, and the sub-displacement direction is determined according to the position information at two adjacent moments; In the case that the included angle is greater than the preset included angle threshold, it is determined that the target vehicle has retrograde behavior; when at least one included angle is less than or equal to the included angle threshold, it is determined that the target vehicle does not have retrograde behavior.

An embodiment of the present disclosure provides a vehicle reverse-travel detection device, including: a first determination part, configured to determine the trajectory information of a target vehicle in a video stream; an acquisition part, configured to acquire the information in the video stream through a preset lane segmentation model Lane information, the lane information includes a driving area and a driving direction corresponding to each of the lanes in the at least one lane; the second determining part is configured to determine whether the target vehicle is in the wrong direction according to the trajectory information and the driving area and driving direction of each lane Behavior.

An embodiment of the present disclosure provides a vehicle reverse-travel detection device, comprising: a memory configured to store an executable computer program; a processor configured to implement the above-mentioned vehicle reverse-travel detection when executing the executable computer program stored in the memory method.

An embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned vehicle retrograde detection method is implemented.

An embodiment of the present disclosure provides a computer program, including computer-readable code, and when the computer-readable code is executed in an electronic device, a processor in the electronic device implements the above-mentioned vehicle retrograde detection method when executed. .

The vehicle reverse-travel detection method provided by the embodiment of the present disclosure determines the trajectory information of the target vehicle in the video stream; obtains the lane information in the video stream through a preset lane segmentation model, and the lane information includes at least one lane corresponding to each of the lanes. According to the trajectory information and the driving area and driving direction of each lane, it is determined whether the target vehicle has a wrong-way behavior. According to the vehicle retrograde detection method provided by the embodiment of the present disclosure, since the trajectory information of the target vehicle is directly obtained according to the video stream, the trajectory information obtained by the embodiment of the present disclosure is more accurate compared with the solution of manually judging the driving direction of the vehicle in the traditional technology. The real-time performance is improved; after obtaining the trajectory information, combined with the lane information of the current lane in the video stream, the detection result of whether the target vehicle has retrograde behavior can be directly obtained, which improves the detection efficiency of vehicle retrograde detection.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the embodiments of the present disclosure.

FIG. 1 is an optional schematic flowchart of a vehicle retrograde detection method provided by an embodiment of the present disclosure;

FIG. 2 is an optional schematic flow chart of the method for detecting reverse movement of a vehicle provided by an embodiment of the present disclosure;

FIG. 3 is an optional schematic flow chart of the method for detecting reverse movement of a vehicle provided by an embodiment of the present disclosure;

FIG. 4 is an optional schematic flow chart of the vehicle retrograde detection method provided by the embodiment of the present disclosure;

FIG. 5 is an optional schematic flowchart of the method for detecting the reverse movement of a vehicle provided by an embodiment of the present disclosure;

FIG. 6 is an optional schematic flow chart of the method for detecting reverse movement of a vehicle provided by an embodiment of the present disclosure;

FIG. 7 is an optional schematic flow chart of the method for detecting reverse movement of a vehicle provided by an embodiment of the present disclosure;

FIG. 8 is an optional system schematic diagram of the vehicle retrograde detection system provided by the embodiment of the present disclosure;

FIG. 9 is a schematic diagram of an optional vehicle detection and tracking provided by an embodiment of the present disclosure;

10A is a schematic diagram of a lane before segmentation in a lane segmentation process according to an embodiment of the present disclosure;

10B is a schematic diagram of a divided lane in a lane division process according to an embodiment of the present disclosure;

FIG. 11 is a schematic diagram of an optional lane direction estimation provided by an embodiment of the present disclosure;

12A is a schematic diagram of a retrograde start image frame of a retrograde behavior of a vehicle according to an embodiment of the present disclosure;

FIG. 12B is a schematic diagram of an image frame in a retrograde motion of a vehicle retrograde behavior according to an embodiment of the present disclosure;

12C is a schematic diagram of a retrograde end image frame of a retrograde behavior of a vehicle according to an embodiment of the present disclosure;

FIG. 12D is a schematic diagram of a vehicle detail image frame of a retrograde behavior of a vehicle according to an embodiment of the present disclosure;

FIG. 13 is a schematic diagram of the composition and structure of a vehicle retrograde detection device provided by an embodiment of the present disclosure;

FIG. 14 is a schematic diagram of a hardware entity of a device according to an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations of the present disclosure, and those skilled in the art will not All other embodiments obtained under the premise of creative work fall within the protection scope of the present disclosure.

FIG. 1 is an optional schematic flow chart of a vehicle retrograde detection method provided by an embodiment of the present disclosure, which will be described in conjunction with the steps shown in FIG. 1 .

S101. Determine track information of a target vehicle in a video stream.

In some embodiments of the present disclosure, the embodiments of the present disclosure may acquire a real-time video stream of the current road through an image acquisition device, and identify a target vehicle in the video stream in real time through a target recognition technology. In the embodiment of the present disclosure, there may be one or more target vehicles identified in the real-time video stream.

In some embodiments of the present disclosure, the video stream is a large number of image frames arranged in time series. According to the hardware factors and configuration information of the image capture device, the current road can be obtained in real time according to the frame rate corresponding to the hardware factors and configuration information. image frame. In order to reduce the amount of calculation and improve the real-time performance of detection, the corresponding image frames to be detected can be extracted from a large number of image frames in the video stream according to the preset image extraction frequency, and the trajectory information of the target vehicle can be obtained according to the image frames to be detected. .

In some embodiments of the present disclosure, S101 may include: performing vehicle identification on the image frame (or the image frame to be detected) in the video stream, and in the case of identifying the vehicle, taking the vehicle as the target vehicle, and using the vehicle as the target vehicle. The image frame of the identified vehicle is used as the initial image frame corresponding to the target vehicle, and the relative position of the target vehicle in the initial image frame is obtained. For several image frames after the initial image frame, the relative position of the target vehicle in each image frame is continuously acquired in chronological order until the target vehicle disappears in the image frames in the video stream. An image frame that appears as the end image frame. By sequentially connecting the relative positions of the target vehicle in the image frame, the trajectory information of the target vehicle can be obtained, and the driving direction of the target vehicle relative to the image frame can be obtained through the trajectory information.

In some embodiments of the present disclosure, the trajectory information of the target vehicle in the video stream may also be acquired by an optical flow method. The trajectory information can be extracted by the following steps: acquiring a video stream of the target vehicle, extracting multiple video screenshots arranged in time series from the video stream, and extracting the target vehicle in each video screenshot through a preset optical flow model According to the optical flow characteristics obtained, the trajectory information of the target vehicle is determined.

S102. Acquire lane information in the video stream through a preset lane segmentation model; the lane information includes a driving area and a driving direction corresponding to each of the at least one lane.

In some embodiments of the present disclosure, the lane information includes a driving area and a driving direction of at least one lane included in the current road in the video stream. The video stream may include multiple lanes, the lane information includes the driving area and the driving direction corresponding to each lane, the area corresponding to each lane is a relative area relative to the image frame in the video stream, and each lane corresponds to The driving direction of is relative to the image frame in the video stream. Acquiring the lane information in the video stream can be achieved in the following ways:

(1) Lane information corresponding to all image acquisition devices and each image acquisition device is pre-stored in the database, and the corresponding lane information can be searched in the database by acquiring the device identification of the image acquisition device.

(2) The relative positions of all image capture devices and each image capture device are pre-stored in the database. By obtaining the device identifiers of the image capture devices, the corresponding relative positions can be searched in the database, and according to the relative positions The map obtains the lane information around the image acquisition device in the real scene.

(3) Extract any image frame in the video stream, perform scene analysis on the image frame, obtain the regional segmentation results and road direction signs of each lane in the image through a preset lane segmentation model, and then determine each lane The corresponding area and direction, that is, the lane information.

S103. Determine whether the target vehicle has a retrograde behavior according to the trajectory information and the driving area and driving direction of each lane.

In some embodiments of the present disclosure, the driving direction of the target vehicle relative to the image frame is obtained through trajectory information, and, according to the driving direction of the current lane relative to the image frame, the distance between the driving direction and the driving direction of the target lane is When the angle difference is greater than the preset angle threshold, it is determined that the target vehicle has a wrong-way behavior; when the angle difference between the driving direction and the driving direction of the target lane is less than or equal to the preset angle threshold, it is determined that the target vehicle does not Retrograde behavior exists.

In some embodiments of the present disclosure, after step 103, the method further includes, in the case where it is determined that the target vehicle has a retrograde behavior, also according to the start image frame and the end image frame corresponding to the trajectory information of the target vehicle, in the A corresponding part of the video is intercepted from the video stream and stored as an archived video of the retrograde behavior corresponding to the target vehicle. Wherein, while detecting the target vehicle, the license plate number of the target vehicle can also be obtained, and in the process of storing the archived video, the license plate number can be used as an index value of the archived video.

It can be known from the above exemplary implementation of FIG. 1 in the embodiment of the present disclosure that the embodiment of the present disclosure determines the trajectory information of the target vehicle in the video stream; obtains the lane information in the video stream through a preset lane segmentation model; information to determine whether the target vehicle has a wrong-way behavior. According to the vehicle retrograde detection method provided by the embodiment of the present disclosure, since the trajectory information of the target vehicle is directly obtained according to the video stream, the trajectory information obtained by the embodiment of the present disclosure is more accurate compared with the traditional method of manually judging the driving direction of the vehicle, and The real-time performance is improved; after obtaining the trajectory information, combined with the lane information of the current lane in the video stream, the detection result of whether the target vehicle has retrograde behavior can be directly obtained, which improves the detection efficiency of vehicle retrograde detection.

FIG. 2 is an optional schematic flowchart of the vehicle retrograde detection method provided by an embodiment of the present disclosure. Based on FIG. 1 , S103 shown in FIG. 1 may include S201 to S204 , which will be described in conjunction with the steps shown in FIG. 2 .

S201. Determine the displacement direction of the target vehicle according to the position information of the target vehicle at multiple times in the trajectory information.

In an embodiment of the present disclosure, the image frame in which the target vehicle appears in the video stream is taken as the start image frame, and the image frame in which the target vehicle appears in the video stream is taken as the end image frame. By analyzing the video stream The track information can be obtained from the starting image frame in the to multiple image frames in the ending image frame (including the starting image frame and the ending image frame). Wherein, each image frame of the plurality of image frames can be input into the vehicle identification model, and the position information of the target vehicle in the image frame in each image frame can be obtained, and the position information is used to represent the position of the target vehicle in the image frame. relative position.

In an embodiment of the present disclosure, the position information used to represent the relative position of the target vehicle in the image frame may include at least one of the following: position information of the detection frame corresponding to the target vehicle, and wheel key point information corresponding to the target vehicle . Wherein, the detection frame position information can be at least one of the following: the relative position of each vertex of the detection frame, the relative position of the center point of the detection frame, the relative position of the middle point of the bottom edge of the detection frame, and the relative position of the center of the circumcircle of the detection frame.

In an embodiment of the present disclosure, the displacement direction of the target vehicle can be determined according to the position information of the target vehicle at multiple times in the trajectory information in the following manner:

(1) Obtain the position information corresponding to the start image frame with the earliest time among the position information at the multiple times, and the position information corresponding to the end image frame with the latest time, according to the position information corresponding to the start image frame and the end image A vector is obtained from the position information corresponding to the frame, and the direction of the vector is the displacement direction of the target vehicle.

(2) Obtaining the position information corresponding to the image frames at two adjacent times in the position information at the multiple times, multiple sub-displacement directions can be obtained, and the average vector of the multiple sub-displacement directions can be used as the displacement direction of the target vehicle. For example, if there is position information at N times, N-1 sub-displacement directions can be obtained according to the position information corresponding to the image frames at two adjacent times, and the average vector of the N-1 sub-displacement directions is used as the displacement direction of the target vehicle.

S202: Determine the target lane where the target vehicle is located according to the position information of the target vehicle at multiple times and the driving area of each lane.

In an embodiment of the present disclosure, when the image acquisition device is initialized, image segmentation is performed on any image frame in the currently captured video stream, and then the lane area of each lane can be obtained, that is, each lane can be obtained. Relative area in the image frame. After acquiring the position information corresponding to the target vehicle, according to the position information, a target lane to which the target vehicle belongs can be determined in at least one lane in the lane information.

S203 , based on whether there is a deviation between the driving direction of the target lane and the displacement direction of the target vehicle, determine whether the target vehicle has a retrograde behavior.

In an embodiment of the present disclosure, the driving direction is a standard direction prescribed by a traffic law. According to the target lane in which the target vehicle is located obtained in S202, the driving direction corresponding to the target lane can be obtained.

In an embodiment of the present disclosure, the above S203 can be implemented by the following method: in the case that the angle difference between the displacement direction and the driving direction of the target lane is greater than a preset angle threshold, it is determined that the target vehicle has a wrong-way behavior; When the angle difference between the displacement direction and the driving direction of the target lane is less than or equal to the preset angle threshold, it is determined that the target vehicle does not have a wrong-way behavior.

In an embodiment of the present disclosure, the above S203 can also be implemented by the following method: obtaining at least one included angle according to the included angle between at least one sub-displacement direction corresponding to the displacement direction of the target vehicle and the driving direction of the target lane; wherein, the displacement The direction includes at least one sub-displacement direction, and the sub-displacement direction is determined according to the position information at two adjacent moments; in the case that each included angle is greater than the preset included angle threshold, it is determined that the target vehicle has retrograde behavior; When the included angle is less than or equal to the included angle threshold, it is determined that the target vehicle does not have a retrograde behavior.

It can be seen from the above exemplary implementation of FIG. 2 in the embodiment of the present disclosure that the relative position of the image frame of the target vehicle in the video stream can be accurately obtained according to the position information corresponding to multiple times in the trajectory information of the target vehicle in the embodiment of the present disclosure, By determining the displacement direction of the target vehicle through the relative position, the accuracy and accuracy of the displacement direction can be improved; since the target lane where the vehicle is located is obtained, and the driving direction of the target lane corresponding to the target lane is determined, there can be different driving directions in the current scene. In the case of multiple lanes in the same direction, it is still possible to accurately determine whether the vehicle is running in the wrong direction, and the application range of the embodiments of the present disclosure can also be improved while improving the detection accuracy of the vehicle in the wrong direction.

FIG. 3 is an optional schematic flowchart of the vehicle retrograde detection method provided by the embodiment of the present disclosure. Based on FIG. 1 or FIG. 2 , and based on FIG. 1 as an example, S102 shown in FIG. 1 can be updated to S301, which will be combined with FIG. 3 The steps shown are explained.

S301. Acquire lane information in a video stream through a lane segmentation model in response to a lane information acquisition condition.

In an embodiment of the present disclosure, the above lane information acquisition conditions include at least one of the following: (1) when the image acquisition device rotates; (2) when the image acquisition device is initialized; wherein, the image acquisition device It is used to obtain the image information in the current real scene and output the video stream. It should be noted that, when any one of the above lane information acquisition conditions is triggered, the lane information in the video stream is acquired through the lane segmentation model.

Wherein, the rotation of the image acquisition device can be detected by the following methods: extracting multiple image frames from the video stream; the multiple image frames include at least the current image frame and the historical image frames adjacent to the current image frame; Historical image frames determine whether the image capture device has rotated. Further, in the embodiment of the present disclosure, the current static sub-image and the historical static sub-image can be respectively intercepted in the current image frame and the historical image frame according to the preset static area, and the current static sub-image and the historical static sub-image can be compared. When the degree of rotation is greater than the preset threshold, it is determined that the image capture device rotates. The preset static area may be determined according to a plurality of image frames acquired during a period of time when the image acquisition device does not rotate. For a road scene, the static area may be the area where environmental objects such as street lamps, trees, and buildings on both sides of the road are located, and the non-static area opposite to the static area is the area on the inner side of the road where vehicles travel.

It should be noted that, in the case of initialization of the image capture device/or rotation of the image capture device, the capture angle of the image capture device for the current scene changes. When the collection angle changes, the track information of the target vehicle collected according to the current collection angle does not match the pre-stored lane information. Therefore, the lane information needs to be re-determined according to the video stream.

In an embodiment of the present disclosure, if the conditions for acquiring the lane information are not met, the lane information is fixed, that is to say, in each process of performing the retrograde detection of the target vehicle, according to the current collection The track information of the target vehicle collected from the angle matches the pre-stored lane information, and the lane information can be directly searched in the database without re-determining the lane information according to the video stream.

It can be seen from the above exemplary implementation of FIG. 3 in the embodiment of the present disclosure that when the image capture device is initialized in the embodiment of the present disclosure/or the image capture device rotates, new lane information is re-determined through the video stream, and the new lane information can be Accurately detect the trajectory information of the target vehicle collected by the current image acquisition device, and then determine whether the vehicle is running in the wrong direction. While improving the accuracy of the vehicle retrograde detection, it can also improve the application range of the above-mentioned vehicle retrograde detection.

FIG. 4 is an optional schematic flowchart of the vehicle retrograde detection method provided by the embodiment of the present disclosure. Based on FIG. 3 , S301 shown in FIG. 3 can be updated to S401 to S403 , which will be described in conjunction with the steps shown in FIG. 4 .

S401. In response to the lane information acquisition condition, input an image frame in the video stream into a preset lane segmentation model to obtain a segmentation result of the image frame, where the segmentation result includes a driving area corresponding to each lane in at least one lane.

In an embodiment of the present disclosure, in response to the lane information acquisition condition, an image frame is acquired in the video stream, the image frame is respectively input into a preset lane segmentation model, and a segmentation result corresponding to the image frame is acquired , the segmentation result includes the driving area corresponding to each lane in the image frame, that is, the relative area of each lane in the image frame.

S402. Acquire sample trajectories corresponding to multiple sample vehicles in each lane of at least one lane according to the sample trajectory acquisition rule.

In an embodiment of the present disclosure, the sample trajectory acquisition rule includes at least one of the following: acquiring the sample trajectory within a preset time interval (ie, at a preset time interval) after the time corresponding to the image frame; The number of sample trajectories. When the number of sample trajectories reaches the preset number, the acquisition of sample trajectories is stopped. Wherein, each lane may include a sample track corresponding to at least one sample vehicle, wherein one sample vehicle corresponds to one sample track.

In an embodiment of the present disclosure, S402 may include: during the acquisition of sample trajectories for the video stream, the sample trajectories in the current video stream are always acquired, and the number of acquired sample trajectories is counted in real time, and when the number of sample trajectories reaches a predetermined number When the number is set, the acquisition of sample trajectories is stopped.

In an embodiment of the present disclosure, S402 may further include: during the acquisition of the sample trajectory of the video stream, the sample trajectory in the current video stream will always be acquired, and when the time interval after the corresponding moment of the image frame arrives, stop Acquisition of sample trajectories.

In an embodiment of the present disclosure, S402 may further include: in the acquisition of sample trajectories for the video stream, if the counted number of sample trajectories does not reach the preset number when the time interval after the time corresponding to the image frame arrives, Then, it is determined that the sample trajectory acquisition fails, and S402 is executed again.

S403: Determine the driving direction corresponding to each lane according to the driving area corresponding to each lane and the sample trajectories corresponding to the plurality of sample vehicles in the driving area corresponding to each lane.

In an embodiment of the present disclosure, according to the sample trajectory of each sample vehicle, the driving direction corresponding to each sample vehicle may be determined. For multiple sample vehicles contained in each lane, the driving directions corresponding to multiple sample vehicles in each lane can be obtained, and the average value of the driving directions of multiple sample vehicles in any lane can be used as the driving direction corresponding to the lane.

For example, for the first lane of the plurality of lanes, in the driving area corresponding to the first lane, vehicle A, vehicle B and vehicle C can be found, wherein the driving direction D1 of vehicle A can be known according to the sample trajectory of vehicle A, The driving direction D2 of vehicle B can be known from the sample trajectory of vehicle B, and the driving direction D3 of vehicle C can be known from the sample trajectory of vehicle C. The driving direction corresponding to the first lane is the average value of D1, D2, and D3. If the unit of the direction is an angle, when D1 is 50 degrees, D2 is 60 degrees, and D3 is 70 degrees, the driving direction corresponding to the first lane is 60 degrees; if the unit of the direction is a vector, in D1 is (0.7 , 0.7), when D2 is (0.5, 0.87), and D3 is (0.87, 0.5), the driving direction corresponding to the first lane is (0.7, 0.7).

It can be seen from the above exemplary implementation of FIG. 4 in the embodiment of the present disclosure that the embodiment of the present disclosure determines the driving direction corresponding to each lane according to the area corresponding to each lane and the sample trajectories corresponding to the multiple sample vehicles in each lane. Analyze multiple sample trajectories in each lane in the current scene, and determine the driving direction of each lane according to the obtained driving directions of multiple sample vehicles in each lane, which can accurately estimate the lane driving in the current scene in any scene. direction, which indirectly improves the accuracy of vehicle retrograde detection.

FIG. 5 is an optional schematic flowchart of the vehicle retrograde detection method provided by the embodiment of the present disclosure. Based on FIG. 4 , S403 shown in FIG. 4 can be updated to S501 to S502 , which will be described in conjunction with the steps shown in FIG. 5 .

S501. Determine the trajectory directions corresponding to the plurality of sample vehicles in the driving area corresponding to each lane according to the starting positions and ending positions corresponding to the plurality of sample vehicles in the driving area corresponding to each lane.

In an embodiment of the present disclosure, for a sample trajectory of each sample vehicle, a start image frame and an end image frame corresponding to the sample trajectory are obtained, and a start position is obtained in the start image frame, and in the end image The end position is obtained in the frame, and the trajectory direction corresponding to the sample trajectory can be determined according to the start position and the end position.

For example, for the first lane of the multiple lanes, in the area corresponding to the first lane, vehicle A and vehicle B can be found, wherein the starting position and ending position of vehicle A can be known according to the sample trajectory of vehicle A, According to the sample trajectory of vehicle B, the starting position and ending position of vehicle B can be known. The trajectory direction of the vehicle A is , and the trajectory direction of the vehicle B is .

S502: Determine a driving direction corresponding to each lane in at least one lane according to the trajectory directions corresponding to the plurality of sample vehicles in the driving area corresponding to each lane.

In an embodiment of the present disclosure, the average value of the trajectory directions corresponding to a plurality of sample vehicles in each lane is used as the driving direction corresponding to the lane.

It can be seen from the above exemplary implementation of FIG. 5 in the embodiment of the present disclosure that the embodiment of the present disclosure determines the trajectory directions corresponding to multiple sample vehicles in each lane through the starting position and the ending position of each sample trajectory, and then determines each trajectory direction. For the driving direction corresponding to one lane, since the starting position and the ending position are used, the prediction accuracy of the corresponding trajectory direction of each sample vehicle can be improved while reducing the amount of calculation; since the target lane is determined according to the trajectory directions corresponding to multiple sample vehicles The driving direction of the target lane is eliminated, and the error of the estimated driving direction of the target lane caused by the abnormal driving of individual vehicles is eliminated, thereby improving the accuracy of lane risk estimation.

FIG. 6 is an optional schematic flowchart of the vehicle retrograde detection method provided by an embodiment of the present disclosure. Based on FIG. 5 and other above-mentioned embodiments, S502 shown in FIG. 5 may be updated to S601 to S602, which will be combined with steps are explained.

S601. Perform normalization processing on each of the multiple track directions to obtain a track vector corresponding to each track direction.

In an embodiment of the present disclosure, S601 includes: determining a track modulo length corresponding to each track direction, and performing normalization processing on each track direction according to the track modulo length of each track direction, so as to obtain each track direction the corresponding trajectory vector. Among them, the trajectory direction is determined by the starting trajectory coordinate point and the ending trajectory coordinate point. First obtain the square sum of the starting trajectory coordinate point and the ending trajectory coordinate point in the trajectory direction, and then take the square root of the square sum to obtain the trajectory direction. For the corresponding trajectory modulo length, the ratio of the starting trajectory coordinate point, the ending trajectory coordinate point and the trajectory modulo length respectively is taken as the trajectory vector corresponding to the trajectory direction.

For example, for a trajectory direction, first determine the trajectory modulo length corresponding to the trajectory direction, and normalize the trajectory direction according to the trajectory modulo length, and then the trajectory vector can be obtained as .

S602. Determine the vector sum of multiple trajectory vectors corresponding to each lane as the driving direction corresponding to each lane.

In an embodiment of the present disclosure, after obtaining multiple trajectory vectors corresponding to each lane, for any one of the lanes, the driving direction corresponding to the lane is determined according to the vectors of the multiple trajectory vectors corresponding to the lane and the lane. . In one embodiment, the vector sum may also be normalized, and the normalized vector sum may be used as the driving direction corresponding to the lane.

For example, if there is a second lane, the second lane includes the trajectory vector and , and the driving direction corresponding to the lane is ++. Exemplarily, in the case of , the driving direction corresponding to the second lane is (1.7, 1.7). The obtained driving direction of the second lane may also be normalized, and the normalized driving direction is obtained as (0.7, 0.7).

It can be seen from the above exemplary implementation of FIG. 6 in the embodiment of the present disclosure that the embodiment of the present disclosure normalizes each trajectory direction, and selects the driving direction corresponding to each lane according to the normalized trajectory vector machine. , which can further reduce the amount of data calculation when predicting the driving direction of the target lane. While ensuring the accuracy of the calculation, the estimation efficiency of the driving direction of the target lane can be improved, thereby improving the detection efficiency of the vehicle's wrong-way behavior.

FIG. 7 is an optional schematic flowchart of the vehicle retrograde detection method provided by the embodiment of the present disclosure. Based on FIG. 6 and other above-mentioned embodiments, the method further includes S701 to S702 , which will be described in conjunction with the steps shown in FIG. 7 .

S701. Determine the quantized value of the trajectory offset corresponding to each lane according to the driving direction corresponding to each lane and the multiple trajectory vectors corresponding to each lane.

In one embodiment of the present disclosure, the trajectory offset quantization value is used to represent the degree of dispersion among the plurality of trajectory vectors corresponding to the lanes. The quantized value of the trajectory offset may be the range, the sum of squares of deviations from the mean, the variance, the standard deviation, and the coefficient of variation between the driving direction corresponding to the lane and multiple trajectory vectors.

In this embodiment, when the quantized value of the trajectory offset is extremely poor, the driving direction and the trajectory vector corresponding to the lane in the vector format can be converted into the format of the angle first, and the maximum angle and the minimum angle are taken as the maximum angle and the minimum angle. The difference between the maximum angle and the minimum angle is used as the track offset quantization value. For example, when the driving direction corresponding to the lane is , the trajectory vector 1 is , and the trajectory vector 2 is Spend.

In this embodiment, when the quantized value of the trajectory offset is the standard deviation, the driving direction and the trajectory vector corresponding to the lane in the vector format can be converted into the format of the angle first, and the multiple angles obtained after the conversion can be calculated. standard deviation. For example, if the driving direction corresponding to the lane is , the trajectory vector 1 is , and the trajectory vector 2 is The standard deviation of 45 degrees, 60 degrees and 30 degrees is 12.25.

S702. In the case that the track offset quantization value exceeds a preset precision threshold, re-determine the lane information according to the video stream.

In an embodiment of the present disclosure, the larger the trajectory offset quantization value is, the more discrete the trajectory vectors corresponding to the lane are; the smaller the trajectory offset quantization value is, the more concentrated the trajectory vectors corresponding to the lane are. Correspondingly, when the quantized value of the trajectory offset exceeds the preset accuracy threshold, it means that the trajectory vectors of the sample vehicles in the lane are far apart, that is, there may be multiple trajectory vectors with retrograde behavior in the collected trajectory vectors. , the accuracy of the driving direction corresponding to the generated lane is low, and the lane information needs to be re-determined according to the video stream. In one embodiment, it may jump to the step of "determining lane information according to the video stream" in the above embodiment.

It can be seen from the above-mentioned exemplary implementation of FIG. 7 in the embodiment of the present disclosure that by obtaining the quantized value of the trajectory offset of the lane used to characterize the degree of dispersion among the plurality of trajectory vectors, the embodiment of the present disclosure can reduce the degree of dispersion of the plurality of trajectory vectors when the degree of dispersion of the plurality of trajectory vectors is relatively high. In the case of high, the accuracy of the driving direction corresponding to the determined lane is low, so that the lane information is regenerated, which can avoid the misjudgment of wrong-way behavior caused by the fact that the trajectory of each sample vehicle cannot accurately reflect the lane information in extreme scenarios; In addition, since the determination of the retrograde behavior is performed when the discrete degree of the multiple trajectory vectors is low, the detection accuracy of the retrograde behavior of the vehicle can be improved.

Below, an exemplary application of the embodiments of the present disclosure in a practical application scenario will be described.

The wrong-way of vehicles is a very serious traffic violation. The wrong-way of a small number of vehicles may cause serious traffic accidents, seriously endangering road safety and road traffic efficiency. Automatic retrograde detection has played a very important role in control in the application of traffic police.

In the related art, in the field of traffic, the detection of wrong-way vehicles mainly relies on a large number of cameras set up on the road. First, a large number of surveillance videos are obtained, and then methods such as optical flow and background modeling are used to detect and track vehicle targets, and then use the obtained vehicle trajectories. The information judges whether the vehicle is going in the wrong direction, supplemented by manual to judge the authenticity of the retrograde event. Many wrong-way vehicles can be detected by the above method, but it is still limited by many scenarios. For example, the lane has no marked range or direction, or the surveillance camera is deflected and the angle of the video picture is different from the original. In these scenarios, the above method cannot continue to work.

Therefore, an embodiment of the present disclosure proposes a vehicle reverse-travel detection method, which realizes the monitoring of the rotation state of the camera, automatically divides the lanes, and estimates the correct driving direction of each lane, so as to complete the detection without manual marking of the driving direction and the possible rotation of the camera. Vehicle retrograde detection task.

Through the implementation of the proposed vehicle retrograde detection algorithm, based on panoramic segmentation, lane area recognition in different driving directions is realized, and the driving direction of each area is estimated to replace manual labeling, which is convenient for large-scale deployment; and, due to the introduction of camera rotation judgment , it can restart quickly after the surveillance camera is rotated to avoid false alarms due to changes in the shooting angle; due to the setting of certain filtering logic and filtering thresholds, false alarms caused by a small number of detection errors can be avoided.

The vehicle retrograde detection is a video abnormal event detection system. The input is the video captured by the surveillance camera, and the output is the retrograde vehicle trajectory. The system includes the following parts: a video structuring part, which is configured to structure the input video stream, can detect and track vehicles, and output its driving trajectory for each vehicle, including the detection frame and wheels of the vehicle at each frame. key point. The scene understanding part is configured to input a screenshot of the surveillance video every other period, and determine whether there is a camera rotation compared with the previous input. If the input is for the first time or the rotation occurs, the panoramic segmentation results of different lanes are output, and the lane direction estimation part is called. The lane direction estimation part is configured to input vehicle trajectories in each lane for the next period of time (usually 5 minutes), and take the average of the moving directions as the lane direction estimation result and output. The vehicle reverse-travel detection part is configured to determine whether there is reverse-travel based on the trajectory of each vehicle and the driving direction of the lane to which it belongs. If there is retrograde, output the information of the trajectory. The lane direction is the driving direction of the target lane in the above embodiment.

8 is an optional system schematic diagram of the vehicle reverse-travel detection system provided by the embodiment of the present disclosure, including: a video structuring part 801 , a scene understanding part 802 , a lane direction estimation part 803 , and a vehicle reverse-travel detection part 804 .

In an embodiment of the present disclosure, the input of the video structuring part 801 is the real-time video stream of the surveillance camera, and the output is the driving track of each vehicle, including the detection frame and wheel key points of the vehicle in each frame; wherein, The detection and tracking tool can be used to obtain the vehicle trajectory appearing in the video, extract its detection frame and key point information, and use it as the input of the lane direction estimation part and the vehicle retrograde detection part.

FIG. 9 is a schematic diagram of an optional vehicle detection and tracking provided by an embodiment of the present disclosure. With the detection and tracking tool, a vehicle in each frame of an image in a current video stream can be detected, and a label frame corresponding to the vehicle can be obtained.

In an embodiment of the present disclosure, the input of the scene understanding part 802 is a screenshot of a certain frame of the video stream, and the output is the division of lanes according to the picture, and the panoramic segmentation results of different lanes are output. In addition, you can also judge whether the camera rotates according to the screenshots of the two frames before and after. Among them, the lane segmentation model and the camera rotation model are used, and the lane segmentation is used as the basis for judging the lane to which each vehicle belongs. And it is determined whether the camera is rotated to decide whether to start the lane direction estimation part again.

10A is a schematic diagram of a lane before segmentation in a lane segmentation process provided by an embodiment of the present disclosure, and the lane segmentation model can be used to segment the lane of each frame of image in the current video stream to obtain the area corresponding to the lane in each frame of image . The lane image obtained after segmentation can be as shown in Figure 10B.

In an embodiment of the present disclosure, the input of the lane direction estimation part 803 is the vehicle trajectory information obtained by the video structuring part, and the segmentation map output by the scene understanding part; the output is the average driving direction of each lane. Among them, the lane direction estimation algorithm is used to read the label of the lane to which the center point of the bottom edge of each vehicle detection frame of each frame belongs to on the segmentation map output by the scene understanding part according to the vehicle trajectory information obtained from the video structuring part. And in the same lane, all trajectories from the start to the end are averaged as the result output.

FIG. 11 is a schematic diagram of an optional lane direction estimation provided by an embodiment of the present disclosure, and the lane direction estimation part can estimate the lane direction of each lane in the current video stream.

In one embodiment of the present disclosure, a vector is defined to represent the direction of a certain vehicle trajectory. For the trajectory Tx, let the center point of the bottom edge of the detection frame of the start frame be , and the center point of the bottom edge of the detection frame of the end frame to be , then . Then the average direction of the lane is defined as shown in Equation 1 and Equation 2.

Among them, it represents the speed of the trajectory within 5 minutes of the current lane; it represents the average direction of the lane.

If the lane direction estimation part cannot collect enough trajectories (not less than 5) within 5 minutes of running, or the standard deviation of each trajectory is too large (greater than 20), it is considered that the direction estimation of the lane fails.

In an embodiment of the present disclosure, the input of the retrograde detection part 804 is the vehicle trajectory information obtained by the video structuring part, the segmentation map output by the scene understanding part, and the lane direction output by the direction estimation part; the output is all retrograde trajectories. . The wrong-way detection section 804 is activated after the lane direction estimation section 803 succeeds.

Among them, since there may be a very small amount of detection frame offset in the structured part of the video, we define a standard for the retrograde trajectory:

For the trajectory Tx, if there are frame numbers frame1, frame2, and frame3, the center points of the bottom edge of the vehicle detection frame at three moments are (x1, y1), (x2, y2), (x3, y3) respectively. Then the displacement direction between them is . If there is a sum, and the included angle of the sum is greater than 120°, and ||>50, ||>50, it is considered that there is retrograde.

In an embodiment of the present disclosure, the vehicle retrograde detection part can output the trajectory information of the target vehicle with retrograde behavior, wherein, multiple image frames corresponding to the retrograde behavior can be output. For example, the retrograde behavior of the target vehicle may be as shown in FIGS. 12A to 12D , wherein, FIG. 12A may be the image frame of the retrograde beginning in the retrograde behavior; FIG. 12B may be the image frame of the retrograde in the retrograde behavior; FIG. 12C may be The image frame of the end of the retrograde movement in the retrograde behavior; FIG. 12D can be a detailed image frame of the vehicle during the retrograde movement in the retrograde behavior.

According to the vehicle retrograde detection method and system provided by the embodiments of the present disclosure, the following technical effects can be achieved: (1) Real-time early warning: the traffic police can use this system to detect dangerous retrograde behavior in time, so as to dispatch police forces to stop the behavior, which can reduce risk of accident. (2) Responsibility after the event: The traffic police can use this system to discover the missed retrograde behavior after the event as the basis for a fine.

FIG. 13 is a schematic structural diagram of a vehicle reverse-travel detection device provided by an embodiment of the present disclosure. As shown in FIG. 13 , the vehicle reverse-travel detection device 1300 includes a first determination part 1301 , an acquisition part 1302 and a second determination part 1303 , in:

The first determining part 1301 is configured to determine the track information of the target vehicle in the video stream;

The obtaining part 1302 is configured to obtain lane information in the video stream through a preset lane segmentation model, where the lane information includes a driving area and a driving direction corresponding to each of the at least one lane;

The second determining part 1303 is configured to determine whether the target vehicle has a wrong-way behavior according to the track information and the driving area and driving direction of each lane.

In an embodiment of the present disclosure, the second determining part 1303 is further configured to determine the displacement direction of the target vehicle according to the position information of the target vehicle at multiple times in the trajectory information; The location information and the driving area of each lane are used to determine the target lane where the target vehicle is located; based on whether there is a deviation between the driving direction of the target lane and the displacement direction of the target vehicle, it is determined whether the target vehicle has retrograde behavior.

In an embodiment of the present disclosure, the acquiring part 1302 is further configured to acquire lane information in the video stream through the lane segmentation model in response to the lane information acquisition condition; the lane information acquisition condition includes at least one of the following: in the image acquisition device In the case of rotation; in the case of initialization of the image acquisition device; wherein, the image acquisition device is used to acquire the image information in the current real scene and output the video stream.

In an embodiment of the present disclosure, the acquiring part 1302 is further configured to extract multiple image frames from the video stream; the multiple image frames at least include the current image frame and the historical image frames adjacent to the current image frame; Frame and historical image frames determine whether the image capture device is rotated.

In an embodiment of the present disclosure, the acquiring part 1302 is further configured to input an image frame in the video stream into the lane segmentation model, and obtain a segmentation result of the image frame, where the segmentation result includes the correspondence between each lane in the at least one lane. According to the sample trajectory acquisition rules, the sample trajectories corresponding to multiple sample vehicles in the driving area corresponding to each lane are obtained; according to the driving area corresponding to each lane and the driving area corresponding to each lane. The sample trajectories of , determine the driving direction corresponding to each lane.

In an embodiment of the present disclosure, the sample trajectory includes the starting position and the ending position of the sample vehicle; the acquiring part 1302 is further configured to be based on the starting position and ending position corresponding to the plurality of sample vehicles in the driving area corresponding to each lane position, determine the trajectory directions corresponding to multiple sample vehicles in the driving area corresponding to each lane; determine the driving direction corresponding to each lane in at least one lane according to the trajectory directions corresponding to multiple sample vehicles in the driving area corresponding to each lane .

In an embodiment of the present disclosure, the sample trajectory acquisition rule includes at least one of the following: acquiring sample trajectories at preset time intervals; counting the number of acquired sample trajectories, and stopping when the number of sample trajectories reaches a preset number Acquisition of sample trajectories.

In an embodiment of the present disclosure, the acquisition part 1302 is further configured to perform normalization processing on each of the multiple trajectory directions to obtain a trajectory vector corresponding to each trajectory direction; A vector sum of multiple trajectory vectors is determined as the direction corresponding to each lane.

In an embodiment of the present disclosure, the second determining part 1303 is further configured to determine the quantized value of the trajectory offset corresponding to each lane according to the driving direction corresponding to each lane and the plurality of trajectory vectors corresponding to each lane ; wherein, the track offset quantization value is used to represent the degree of dispersion between the multiple track vectors corresponding to the lane; when the track offset quantization value exceeds the preset accuracy threshold, the lane information is re-determined according to the video stream.

In an embodiment of the present disclosure, the second determining part 1303 is further configured to obtain at least one included angle according to the included angle between at least one sub-displacement direction corresponding to the displacement direction of the target vehicle and the driving direction of the target lane; wherein, The displacement direction includes at least one sub-displacement direction, and the sub-displacement direction is determined according to the position information at two adjacent moments; in the case that each included angle is greater than the preset included angle threshold, it is determined that the target vehicle has retrograde behavior; When at least one included angle is less than or equal to the included angle threshold, it is determined that the target vehicle does not have a retrograde behavior.

It should be pointed out here that the descriptions of the above apparatus embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to the method embodiments. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the descriptions of the method embodiments of the present disclosure for understanding.

It should be noted that, in the embodiments of the present disclosure, if the above-mentioned vehicle retrograde detection method is implemented in the form of a software functional part and sold or used as an independent product, it may also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present disclosure may be embodied in the form of software products that are essentially or contribute to related technologies. The computer software products are stored in a storage medium and include several instructions to make The terminal (which may be a smart phone with a camera, a tablet computer, etc.) executes all or part of the methods described in the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), magnetic disk or optical disk and other media that can store program codes. As such, embodiments of the present disclosure are not limited to any particular combination of hardware and software.

In the embodiments of the present disclosure and other embodiments, a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course, a unit, a module or a non-modularity.

Correspondingly, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in any of the vehicle retrograde detection methods in the foregoing embodiments.

Correspondingly, in the embodiment of the present disclosure, a chip is also provided, the chip includes a programmable logic circuit and/or program instructions, and when the chip is running, it is configured to implement any one of the above embodiments. Describe the steps in the vehicle retrograde detection method.

Correspondingly, in an embodiment of the present disclosure, a computer program product is also provided. When the computer program product is executed by a processor of a terminal, the computer program product is configured to implement any one of the vehicle retrograde detection methods in the foregoing embodiments. step.

Based on the same technical concept, the embodiments of the present disclosure provide a device configured to implement the vehicle retrograde detection method described in the above method embodiments. FIG. 14 is a schematic diagram of a hardware entity of a device provided by an embodiment of the present disclosure. As shown in FIG. 14 , the device 1400 includes a memory 1410 and a processor 1420 , and the memory 1410 stores a computer that can run on the processor 1420 A program, when the processor 1420 executes the program, implements the steps in any of the vehicle retrograde detection methods in the embodiments of the present disclosure.

The memory 1410 is configured to store instructions and applications executable by the processor 1420, and can also cache data to be processed or processed by the processor 1420 and various parts of the terminal (for example, image data, audio data, voice communication data and video communication data). data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

When the processor 1420 executes the program, it implements the steps of any one of the above-mentioned methods for detecting the wrong direction of a vehicle. The processor 1420 generally controls the overall operation of the device 1400.

The above-mentioned processor can be a special purpose integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic At least one of Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), Central Processing Unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor. It can be understood that, the electronic device implementing the function of the above processor may also be other, which is not limited in the embodiment of the present disclosure.

The above-mentioned computer-readable storage medium/memory can be a read-only memory (Read Only Memory, ROM), a programmable read-only memory (Programmable Read-Only Memory, PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory) Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Random Access Memory (FRAM), Flash Memory (Flash Memory), Magnetic Surface Memory, optical disk, or memory such as Compact Disc Read-Only Memory (CD-ROM); it can also be various terminals including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital Assistant etc.

It should be pointed out here that the descriptions of the above storage medium and terminal embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to the method embodiments. For technical details not disclosed in the embodiments of the storage medium and terminal of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

It is to be understood that reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic associated with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present disclosure, the size of the sequence numbers of the above-mentioned processes does not imply the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, rather than the embodiments of the present disclosure. implementation constitutes any limitation. The above-mentioned serial numbers of the embodiments of the present disclosure are for description, and do not represent the advantages or disadvantages of the embodiments.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

In the several embodiments provided in the present disclosure, it should be understood that the disclosed terminal and method may be implemented in other manners. The terminal embodiments described above are illustrative. For example, the division of the units is a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another system, or some features can be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of. The unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present disclosure.

In addition, each functional unit in each embodiment of the present disclosure may be all integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration The unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.

Alternatively, if the above-mentioned integrated units in the embodiments of the present disclosure are implemented in the form of software functional parts and sold or used as independent products, they may also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present disclosure may be embodied in the form of software products that are essentially or contribute to related technologies. The computer software products are stored in a storage medium and include several instructions to make The device automated test line performs all or part of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

The methods disclosed in the several method embodiments provided in the present disclosure can be combined arbitrarily without conflict to obtain new method embodiments.

The features disclosed in several methods or terminal embodiments provided in the present disclosure can be combined arbitrarily without conflict to obtain new method embodiments or terminal embodiments.

The above is the implementation of the embodiments of the present disclosure, but the protection scope of the embodiments of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes within the technical scope disclosed by the embodiments of the present disclosure. Or alternatives, all should be covered within the protection scope of the embodiments of the present disclosure. Therefore, the protection scope of the embodiments of the present disclosure should be subject to the protection scope of the claims.

Industrial Applicability

Embodiments of the present disclosure provide a method, device, device, computer-readable storage medium, and computer program product for vehicle reverse-travel detection; wherein, the vehicle reverse-travel detection method includes: determining trajectory information of a target vehicle in a video stream; passing a preset lane The segmentation model obtains the lane information in the video stream, and the lane information includes the driving area and the driving direction corresponding to each of the lanes in at least one lane; according to the trajectory information and the driving area and driving direction of each lane, determine whether the target vehicle has a wrong-way behavior . Through the above solution, the detection accuracy and detection efficiency of vehicle retrograde detection can be improved.

Claims

A vehicle retrograde detection method, comprising:

Determine the trajectory information of the target vehicle in the video stream;

Acquire lane information in the video stream through a preset lane segmentation model, where the lane information includes a driving area and a driving direction corresponding to each of the at least one lane;

Whether the target vehicle has a wrong-way behavior is determined according to the trajectory information and the driving area and driving direction of each of the lanes.
The method according to claim 1, wherein the determining whether the target vehicle has a wrong-way behavior according to the trajectory information and the driving area and driving direction of each of the lanes comprises:

Determine the displacement direction of the target vehicle according to the position information of the target vehicle at multiple times in the trajectory information;

Determine the target lane where the target vehicle is located according to the position information of the target vehicle at multiple times and the driving area of each of the lanes;

Based on whether there is a deviation between the driving direction of the target lane and the displacement direction of the target vehicle, it is determined whether the target vehicle has a wrong-way behavior.
The method according to claim 2, wherein, based on whether there is a deviation between the driving direction of the target lane and the displacement direction of the target vehicle, determining whether the target vehicle has a wrong-way behavior comprises:

At least one included angle is obtained according to the included angle between at least one sub-displacement direction corresponding to the displacement direction of the target vehicle and the driving direction of the target lane; wherein the displacement direction includes at least one sub-displacement direction, and each of the The sub-displacement direction is determined according to the position information at two adjacent moments;

In the case that each of the included angles is greater than a preset included angle threshold, determine that the target vehicle has a retrograde behavior;

When at least one of the included angles is less than or equal to the included angle threshold, it is determined that the target vehicle does not have a wrong-way behavior.
The method according to any one of claims 1 to 3, wherein the acquiring lane information in the video stream through a preset lane segmentation model comprises:

Acquiring lane information in the video stream through the lane segmentation model in response to lane information acquisition conditions;

The lane information acquisition conditions include at least one of the following:

When the image acquisition device rotates;

When the image capture device is initialized; wherein, the image capture device is used to acquire image information in the current real scene and output the video stream.
The method according to claim 4, wherein the method for detecting the rotation of the image capture device comprises:

Extract multiple image frames from the video stream; the multiple image frames include at least a current image frame and a historical image frame adjacent to the current image frame;

Whether the image capturing device rotates is determined through the current image frame and the historical image frame.
The method according to any one of claims 1 to 5, wherein the acquiring lane information in the video stream through a lane segmentation model comprises:

Inputting an image frame in the video stream into the lane segmentation model to obtain a segmentation result of the image frame, where the segmentation result includes a driving area corresponding to each of the at least one lane;

Obtain sample trajectories corresponding to a plurality of sample vehicles in the driving area corresponding to each of the lanes according to the sample trajectory acquisition rule;

The driving direction corresponding to each of the lanes is determined according to the driving area corresponding to each of the lanes and the sample trajectories corresponding to a plurality of sample vehicles in the driving area corresponding to each of the lanes.
The method according to claim 6, wherein the sample trajectory includes a starting position and an ending position of the sample vehicle; the driving area corresponding to each of the lanes and the driving area corresponding to each of the lanes The sample trajectories corresponding to a plurality of sample vehicles in the sample trajectories are determined, and the driving direction corresponding to each said lane is determined, including:

According to the starting positions and ending positions corresponding to the plurality of sample vehicles in the driving area corresponding to each of the lanes, determine the trajectory directions corresponding to the plurality of sample vehicles in the driving area corresponding to each of the lanes;

The driving direction corresponding to each of the at least one lane is determined according to the trajectory directions corresponding to a plurality of sample vehicles in the driving area corresponding to each of the lanes.
The method according to claim 6, wherein the sample trajectory acquisition rule includes at least one of the following:

Obtain sample trajectories at preset time intervals;

The number of acquired sample trajectories is counted, and when the number of sample trajectories reaches a preset number, the acquisition of the sample trajectories is stopped.
The method according to claim 7, wherein the driving direction corresponding to each of the at least one lane is determined according to the trajectory directions corresponding to a plurality of sample vehicles in the driving area corresponding to each of the lanes, include:

performing normalization processing on each of the multiple track directions to obtain a track vector corresponding to each of the track directions;

A vector sum of multiple trajectory vectors corresponding to each of the lanes is determined as the driving direction corresponding to each of the lanes.
The method of claim 9, wherein the method further comprises:

According to the driving direction corresponding to each of the lanes and the plurality of trajectory vectors corresponding to each of the lanes, a quantized value of the trajectory offset corresponding to each of the lanes is determined; wherein the quantized value of the trajectory offset is used to represent the degree of dispersion between the multiple trajectory vectors corresponding to the lane;

In the case that the track offset quantization value exceeds a preset precision threshold, the lane information is re-determined according to the video stream.
A vehicle retrograde detection device, comprising:

a first determining part, configured to determine the trajectory information of the target vehicle in the video stream;

an acquiring part, configured to acquire lane information in the video stream through a preset lane segmentation model, where the lane information includes a driving area and a driving direction corresponding to each of the at least one lane;

The second determination part is configured to determine whether the target vehicle has a wrong-way behavior according to the trajectory information and the driving area and driving direction of each of the lanes.
A vehicle retrograde detection device, comprising:

a memory configured to store executable instructions;

A processor, configured to execute the computer program stored in the memory, implements the method of any one of claims 1 to 10.
A computer-readable storage medium storing a computer program, when the computer program is executed by a processor, the method of any one of claims 1 to 10 is implemented.
A computer program, comprising computer-readable codes, when the computer-readable codes are executed in an electronic device, when executed by a processor in the electronic device, the implementation of any one of claims 1 to 10 is realized. Methods.