CN113947743A

CN113947743A - Vehicle illegal driving detection method and device, computer equipment and storage medium

Info

Publication number: CN113947743A
Application number: CN202111240792.6A
Authority: CN
Inventors: 张先炳; 赵兴杰
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2021-10-25
Filing date: 2021-10-25
Publication date: 2022-01-18

Abstract

The present disclosure provides a vehicle peccancy running detection method, device, computer equipment and storage medium, wherein the method comprises the following steps: acquiring a to-be-detected video corresponding to a preset detection area and marking information corresponding to each lane in the preset detection area; identifying at least partial image in a video to be detected, and determining first track information of a target vehicle in the video to be detected; and determining the violation driving information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane.

Description

Vehicle illegal driving detection method and device, computer equipment and storage medium

Technical Field

The disclosure relates to the technical field of computer vision and intelligent traffic, in particular to a method and a device for detecting vehicle illegal running, computer equipment and a storage medium.

Background

According to the road traffic rule, in the case of a plurality of lanes running in the same direction, the large passenger car, the medium-sized truck or the heavy truck is not allowed to enter the left lane for ensuring the driving safety, and in the case of only two lanes running in the same direction, the large passenger car, the medium-sized truck or the heavy truck is allowed to borrow the left lane during overtaking in consideration of the actual situation. In order to increase the management and control force of traffic violation conditions, traffic management departments often adopt a large amount of manpower and material resources to patrol traffic monitoring videos, but the scheme of finding out the violation vehicles through manual patrol is not comprehensive in investigation and low in real-time performance, and a large amount of traffic videos are investigated one by one, so that the investigation efficiency is low; in addition, a large amount of manpower and material resources are input, and the management and control cost is improved, so that a more efficient detection method needs to be developed to detect the illegal lane occupation condition in a traffic scene urgently.

Disclosure of Invention

The embodiment of the disclosure at least provides a method and a device for detecting vehicle illegal running, computer equipment and a storage medium.

In a first aspect, the disclosed embodiment provides a vehicle illegal running detection method, including:

acquiring a to-be-detected video corresponding to a preset detection area and marking information corresponding to each lane in the preset detection area;

identifying at least partial images in the video to be detected, and determining first track information of a target vehicle in the video to be detected;

and determining the violation driving information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane.

In the aspect, images are identified by using a video structured platform and the like, first track information of a target vehicle is determined based on structural information of the target vehicle obtained through identification, then the corresponding marking information (used for judging the illegal lane occupation condition) of a lane marked in a preset detection area is combined, the behavior of the target vehicle is analyzed, whether the target vehicle has the illegal driving condition or not is judged, namely the illegal driving information of the target vehicle, the capability of the existing platforms such as the video structured platform and the like is fully utilized, the target vehicle is structurally detected, the illegal driving detection cost of the vehicle is reduced, meanwhile, the behavior detection of the target vehicle is carried out by combining an identification result, the detection precision and the detection efficiency can be improved, and the human resources can be saved.

In an optional embodiment, the labeling information includes position information of the corresponding lane and label information of the corresponding lane;

the determining of the violation driving information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane comprises the following steps:

determining at least one passing lane through which the target vehicle passes based on the first trajectory information and the position information of each lane;

and determining the violation driving information of the target vehicle based on the label information corresponding to each passing lane.

According to the embodiment, whether the target vehicle runs in the violation driving lane or the overtaking lane can be accurately determined by utilizing the label information corresponding to each passing lane, and whether the target vehicle runs in the violation driving lane or the overtaking lane can be accurately determined by combining the passing lane judgment, namely the violation driving information of the target vehicle is accurately determined.

In an optional embodiment, the determining the illegal driving information of the target vehicle based on the label information corresponding to each passing lane includes:

determining second track information of a target vehicle in a first target lane based on the first track information, wherein the first target lane is a overtaking lane determined based on the label information of each passing lane;

and determining the illegal driving information of the target vehicle based on the second track information.

In the embodiment, whether the target vehicle runs in the first target lane can be accurately determined by using the label information, wherein the first target lane is a specific lane for judging illegal running, for example, the first target lane is a passable lane; under the condition that the target vehicle runs in the first target lane, the running track of the target vehicle in the first target lane, namely the second track information, can be combined to accurately determine whether the target vehicle has violation behaviors such as overtaking and the like, and the violation running information can be accurately determined based on the violation behavior.

and under the condition that the target vehicle is determined to be in a second target lane based on the first track information, determining that the violation driving information comprises the violation driving information of the target vehicle, wherein the second target lane is the violation driving lane determined based on the label information of each passing lane.

In the embodiment, whether the target vehicle runs in the second target lane can be accurately determined by using the label information, and the second target lane is a specific lane for judging the illegal running, namely the illegal running lane; under the condition that the target vehicle runs in the second target lane, the illegal running of the target vehicle can be directly and accurately determined, namely the illegal running information can be more accurately determined.

In an optional embodiment, the labeling information further includes a lane direction; the determining of the illegal driving information of the target vehicle based on the second track information comprises the following steps:

acquiring third track information corresponding to other vehicles on a third target lane in a time period corresponding to the second track information, wherein the third target lane is a legal driving lane determined based on the label information of each passing lane;

determining the violation running information of the target vehicle based on the target overlapping information between the tracks respectively corresponding to the second track information and the third track information; the target overlap information includes overlap information on a lane direction extension line.

In the embodiment, the third target lane can be determined more accurately by using the tag information, the third target lane can be a legal driving lane of the target vehicle, and then the overlapping information between two tracks on the lane direction extension line can be determined by using the second track information and combining with the third track information of other vehicles driving in the third target lane, so that the behavior information of the current target vehicle relative to other vehicles, such as overtaking behaviors and the like, can be accurately determined, and the accurate violation driving information of the target vehicle can be obtained.

In an optional embodiment, the obtaining third trajectory information corresponding to other vehicles in a third target lane in a time period corresponding to the second trajectory information includes:

determining first displacement information of the target vehicle within the first target lane based on the second trajectory information;

and under the condition that the first displacement information meets a preset displacement condition, acquiring third track information corresponding to other vehicles on a third target lane in a time period corresponding to the second track information.

According to the embodiment, after the target vehicle travels a certain distance on the first target lane, the first displacement information exists, that is, the overtaking behavior exists at a high probability, and then the third track information of other vehicles on the third target lane is acquired, so that the operation of acquiring the traveling tracks of other vehicles on the third target lane can be reduced, the memory occupation of equipment is reduced, the data processing efficiency of the equipment is improved, the service life of the equipment is prolonged, and the cost of the equipment is reduced. For example, the target vehicle mistakenly travels to the first target lane, travels for an extremely short distance and then returns to the original legal travel lane, at this time, the target vehicle travels on the first target lane but is not judged to be traveling violating the regulations, so that the third track information corresponding to other vehicles on the third target lane does not need to be acquired any more, and the operation of acquiring the travel tracks of other vehicles on the third target lane can be reduced.

In an optional embodiment, before the determining the violation driving information of the target vehicle based on the target overlap information between the tracks corresponding to the second track information and the third track information, the method further includes:

determining first projection information of the second track information on the lane direction extension line and second projection information of the third track information on the lane direction extension line;

determining target overlap information between the first projection information and the second projection information;

the determining of the violation driving information of the target vehicle based on the target overlapping information between the tracks respectively corresponding to the second track information and the third track information includes:

and determining the illegal driving information of the target vehicle based on the target overlapping information and the first projection information.

In the embodiment, by using the comparison information between the target overlapping information and the first projection information, the behavior information of the current target vehicle, such as the violation behavior, can be accurately judged, and the accurate violation driving information of the target vehicle can be further obtained.

In an optional embodiment, the determining the violation information of the target vehicle based on the target overlapping information and the first projection information includes:

and under the condition that the target overlapping information is positioned at one end of the first projection information and the projection length corresponding to the projection information left after the first projection information is removed from the target overlapping information is greater than a first preset length, determining that the violation driving information comprises the violation driving information of the target vehicle.

In this embodiment, if the target overlapping information is located at one end of the first projection information, and the projection length corresponding to the projection information remaining after the first projection information is removed from the target overlapping information is greater than the first preset length, it can be determined that the target vehicle is traveling on the overtaking lane for a long time in addition to overtaking, so that it can be accurately determined that the target vehicle is traveling violatively on the overtaking lane.

determining that the target vehicle has overtaking behavior under the condition that the target overlapping information is not empty;

and under the condition that the target vehicle has overtaking behaviors and the projection length corresponding to the first projection information is smaller than a second preset length, determining that the violation driving information does not include the violation driving information of the target vehicle.

In this embodiment, since the target overlap information is not empty, it can be determined that the target vehicle has a passing behavior; at this time, if the projection length corresponding to the projection information remaining after the first projection information is removed from the target overlap information is smaller than the second preset length, it can be determined that the target vehicle does not travel on the overtaking lane for a long time except for overtaking, so it can be accurately determined that the target vehicle does not travel illegally on the overtaking lane.

In an optional implementation manner, the identifying at least a partial image in the video to be detected and determining first track information of a target vehicle in the video to be detected includes:

identifying at least partial images in the video to be detected, and determining key points corresponding to the target vehicle in the at least partial images;

and determining first track information of the target vehicle in the video to be detected based on the identified key point corresponding to the target vehicle.

In the embodiment, the identified preset part of the target vehicle is used as a key point, so that the coordinate position of the target vehicle in the preset detection area can be accurately determined, and the accurate first track information is determined by using the accurate coordinate position.

In an optional embodiment, after determining the violation driving information, the method further comprises:

under the condition that the target vehicle is determined to run illegally, generating a running illegal report of the target vehicle;

the violation travel report includes at least one of: the violation driving information of the target vehicle, the structural information of the target vehicle, the starting frame image corresponding to the first track information, the ending frame image corresponding to the first track information, the middle frame image corresponding to the first track information and the target frame image corresponding to the first track information; the target frame image comprises a target vehicle meeting a preset condition.

In the embodiment, the violation running report recorded with more detailed running information is utilized, so that the violation running behavior of the target vehicle can be comprehensively and accurately characterized, related personnel are informed to stop the violation behavior in time, and the risk of traffic accidents is reduced.

In an optional embodiment, the method further comprises the step of determining the structured information of the target vehicle:

identifying vehicles of the images in the video to be detected, determining vehicle images including the target vehicle, and determining vehicle information corresponding to the target vehicle based on the vehicle images;

performing license plate recognition on the vehicle image, determining a license plate image comprising the license plate, and determining license plate information corresponding to the license plate based on the license plate image;

and determining the structural information corresponding to the target vehicle based on the vehicle information and the license plate information.

In the embodiment, the existing platforms such as a video structured platform and the like are fully utilized to detect the video to be detected, the structure of a target vehicle in the video is analyzed, and the vehicle information is determined; the structure of the license plate in the target vehicle is analyzed, the license plate information is determined, the structural information corresponding to the target vehicle can be obtained comprehensively, and the vehicle illegal driving detection cost is reduced.

In a second aspect, an embodiment of the present disclosure further provides a vehicle illegal driving detection device, including:

the system comprises an information acquisition module, a detection module and a control module, wherein the information acquisition module is used for acquiring a to-be-detected video corresponding to a preset detection area and marking information corresponding to each lane in the preset detection area;

the track determining module is used for identifying at least part of images in the video to be detected and determining first track information of a target vehicle in the video to be detected;

and the violation detection module is used for determining violation running information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane.

the violation detection module is used for determining at least one passing lane through which the target vehicle passes based on the first track information and the position information of each lane;

In an optional embodiment, the violation detection module is configured to determine second track information of the target vehicle in a first target lane based on the first track information, where the first target lane is a overtaking lane determined based on the tag information of each passing lane;

In an optional embodiment, the violation detection module is configured to determine that the violation driving information includes information that the target vehicle runs in a violation manner under the condition that the target vehicle is determined to be in a second target lane based on the first track information, where the second target lane is a violation driving lane determined based on the tag information of each passing lane.

In an optional embodiment, the labeling information further includes a lane direction; the violation detection module is used for acquiring third track information corresponding to other vehicles on a third target lane in a time period corresponding to the second track information, wherein the third target lane is a legal driving lane determined based on the label information of each passing lane;

In an optional embodiment, the violation detection module is specifically configured to determine first displacement information of the target vehicle in the first target lane based on the second track information;

In an optional embodiment, the violation detection module is further configured to determine first projection information of the second track information on a lane direction extension line and second projection information of the third track information on the lane direction extension line before determining violation driving information of the target vehicle based on target overlapping information between tracks corresponding to the second track information and the third track information, respectively;

and the violation detection module is used for determining violation running information of the target vehicle based on the target overlapping information and the first projection information.

In an optional implementation manner, the violation detection module is configured to determine that the violation driving information includes information that the target vehicle runs in violation, when the target overlapping information is located at one end of the first projection information, and a projection length corresponding to projection information remaining after the first projection information is removed from the target overlapping information is greater than a first preset length.

In an optional embodiment, the violation detection module is configured to determine that there is overtaking behavior of the target vehicle when the target overlapping information is not empty;

In an optional implementation manner, the track determining module is configured to identify at least a partial image in the video to be detected, and determine a key point corresponding to the target vehicle in the at least a partial image;

In an optional embodiment, the device further comprises a report generation module, which is used for generating a violation running report of the target vehicle under the condition that the violation running information of the target vehicle is determined;

In an optional implementation manner, the apparatus further includes an information determining module, configured to perform vehicle identification on the image in the video to be detected, determine a vehicle image including the target vehicle, and determine vehicle information corresponding to the target vehicle based on the vehicle image;

In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating via the bus when the computer device is operating, the machine readable instructions when executed by the processor performing the steps of the first aspect, or any one of the possible vehicle violation detection methods of the first aspect.

In a fourth aspect, the disclosed embodiments also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, performs the steps of the first aspect, or any one of the possible vehicle illegal running detection methods in the first aspect.

For the description of the effects of the vehicle illegal running detection device, the computer equipment and the storage medium, reference is made to the description of the vehicle illegal running detection method, and the description is omitted here.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

FIG. 1 illustrates a flow chart of a vehicle violation detection method provided by an embodiment of the present disclosure;

fig. 2 shows a display schematic diagram of a frame of image to be detected shot by a shooting device and annotation information corresponding to the image in a vehicle illegal driving detection scene provided by the embodiment of the disclosure;

fig. 3a is a schematic illustration showing a first track information of a target vehicle on a same-direction three-row lane according to an embodiment of the present disclosure;

fig. 3b is a schematic illustration showing another first track information of the target vehicle on the same-direction three-row lane provided by the embodiment of the disclosure;

FIG. 4a is a schematic diagram illustrating a display of target overlapping information provided by an embodiment of the present disclosure;

FIG. 4b is a schematic illustration showing another example of target overlapping information provided by the embodiment of the present disclosure;

FIG. 5 illustrates a display schematic view of a violation travel report provided by an embodiment of the present disclosure;

FIG. 6 illustrates a decision flow diagram for target vehicle violation detection provided by embodiments of the present disclosure;

FIG. 7 is a schematic flow chart illustrating a process for determining whether a passing behavior of a target vehicle exists according to an embodiment of the disclosure;

FIG. 8 illustrates a schematic diagram of a vehicle violation detection device provided by an embodiment of the present disclosure;

fig. 9 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

Furthermore, the terms "first," "second," and the like in the description and in the claims, and in the drawings described above, in the embodiments of the present disclosure are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein.

Reference herein to "a plurality or a number" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. The "left" and "right" referred to herein are both left and right with respect to the direction of forward travel of the vehicle.

According to the road traffic rule, under the condition that a plurality of lanes running in the same direction exist, in order to ensure the driving safety, the large passenger car, the medium-sized truck or the heavy truck is not allowed to enter the left lane to run, and under the condition that only two lanes running in the same direction exist, the large passenger car, the medium-sized truck or the heavy truck is allowed to temporarily borrow the left lane during overtaking considering the actual situation. In order to increase the management and control force of traffic violation conditions, traffic management departments often adopt a large amount of manpower and material resources to patrol traffic monitoring videos, but the scheme of finding out the violation vehicles through manual patrol is not comprehensive in investigation and low in real-time performance, and a large amount of traffic videos are investigated one by one, so that the investigation efficiency is low; in addition, a large amount of manpower and material resources are input, and the management and control cost is improved, so that a more efficient detection method needs to be developed to detect the illegal lane occupation condition in a traffic scene urgently.

Based on the research, the invention provides a vehicle illegal running detection method, which is characterized in that an image is identified by using a video structured platform and the like, the first track information of a target vehicle is determined based on the structural information of the target vehicle obtained through identification, then the behavior of the target vehicle is analyzed by combining with the corresponding marking information (used for judging the illegal occupation condition of the lane) of the lane marked in a preset detection area, whether the target vehicle has the illegal running condition or not is judged, namely the illegal running information of the target vehicle, the capability of the existing platform such as the video structured platform is fully utilized, the structural detection is carried out on the target vehicle, the detection cost of the illegal running of the vehicle is reduced, and meanwhile, the behavior detection is carried out on the target vehicle by combining with the identification result, so that the detection precision and the detection efficiency can be improved, and the human resources can be saved.

The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

For the convenience of understanding of the embodiment, firstly, an application scenario of the method for detecting vehicle illegal driving disclosed by the embodiment of the present disclosure is described in detail, according to a traffic violation driving rule, for example, 2 lanes exist in the same direction, a large passenger car, a medium-sized truck and a heavy truck are not allowed to enter a left lane for driving, but in consideration of actual situations, the large passenger car, the medium-sized truck and the heavy truck are allowed to temporarily borrow the left lane (the left side in the forward driving direction of the vehicle) during overtaking, but the vehicle speed is not lower than 20% of the lowest speed limit value of the lane during overtaking, the vehicle is required to return to the right lane immediately after overtaking, and the vehicle is prohibited to continue to occupy the left lane for driving after overtaking. For another example, if there are more than 3 lanes in the same direction, the large bus, the medium-sized truck and the heavy truck are prohibited from entering the leftmost lane to drive or overtaking in the leftmost lane, the overtaking in the middle lane is allowed, but the speed of the overtaking is not lower than 20% of the lowest speed limit value of the lane, the overtaking is immediately returned to the right lane, and the overtaking is prohibited from continuing to occupy the middle lane. For another example, a large bus, a medium-sized truck, and a heavy truck are prohibited from entering 2 leftmost lanes or using 2 leftmost lanes to overtake, wherein 4 lanes or more are provided in the same direction. The embodiment of the disclosure provides a method for detecting illegal vehicle running, which can detect illegal vehicle running information of vehicles such as large buses, medium-sized and heavy trucks, for example, illegal vehicle lane occupation information.

The following describes in detail a vehicle illegal running detection method disclosed in the embodiments of the present disclosure, and an execution main body of the vehicle illegal running detection method provided in the embodiments of the present disclosure is generally a computer device with certain computing capability. In some possible implementations, the vehicle violation detection method may be implemented by the processor invoking computer readable instructions stored in the memory.

The vehicle illegal running detection method provided by the embodiment of the disclosure is explained by taking an execution main body as a computer device as an example.

Based on the application scenario, referring to fig. 1, a flowchart of a vehicle illegal driving detection method provided by the embodiment of the present disclosure is shown, where the method includes steps S101 to S103, where:

s101: and acquiring the to-be-detected video corresponding to the preset detection area and the marking information corresponding to each lane in the preset detection area.

In this step, the preset detection area may be a shooting area preset on the terminal device, or may be an area where a lane under the shooting area is located. The marking information corresponding to each lane is preset on a preset detection area by a user and can support change operation. The video to be detected can be acquired through the shooting equipment arranged in a specific scene specified by the detection task, and the shooting equipment can be equipment supporting video or image acquisition. A characteristic scene specified by the task, for example, a vehicle driving scene including a plurality of lanes in the same direction is detected.

In this embodiment, the video to be detected includes a video segment to be detected, and may be, for example, a certain video segment captured from a video stream captured by the capturing device, or a video segment with a fixed frame number and obtained at a certain capturing frequency by using the capturing device.

Specifically, as shown in fig. 2, the display diagram is a frame of image to be detected shot by the shooting device in the scene of detecting the vehicle running against regulations, and the display diagram of the annotation information corresponding to the image. The method comprises the following steps: 21 denotes an image to be detected; 22 denotes a preset detection area, which may include a plurality of lanes in the same direction, and/or a plurality of lanes in the same first driving direction and a plurality of lanes in the same second driving direction opposite to the first driving direction (in the case illustrated in fig. 2) within the preset detection area 22; and label information corresponding to each lane, wherein the label information may include position information 231 (indicated by a rectangular dashed line frame) of the lane, label information 232 of the lane, and a lane direction 233 (a direction in which a vehicle is allowed to travel on the lane, such as a lane direction extension line parallel to a lane line between two adjacent lanes in fig. 2).

The position information 231 of the lane may be used to indicate a specific lane area of the lane in the preset detection area, and may include information such as coordinates or an area of the lane area. The lane area may be a closed area surrounded by polygons. Such as a regular quadrilateral as shown in fig. 2, or may be an irregular polygon, and the embodiments of the present disclosure are not particularly limited. Illustratively, detecting vehicle violation travel is detected only within the lane area indicated by the location information 231 of the lane.

The label information 232 for a lane may be used to indicate the lane type of the lane, which may include legal travel type, overtaking type, violation travel type, and the like. Here, the lane type may be set according to a regulation for driving against a regulation violation issued by a traffic management department, or may be set according to other application scenarios, and the embodiment of the present disclosure is not particularly limited. Exemplarily, the lane is arranged in three same-direction rows on a highway, wherein the leftmost lane is a violation driving lane of a truck; the middle lane is a overtaking lane of a large truck, and the rightmost lane is a legal driving lane of the large truck. The label information of the lane may be a number or a character, for example, 01 indicates a legal driving lane, 02 indicates a overtaking lane, and 03 indicates a violation driving lane.

For example, the labeling format corresponding to the labeling information of each lane may be < lane region, i.e. polygon; lane direction, i.e., arrow; lane labels, i.e., labels >, where polygon represents an arbitrary closed polygon of a lane region, may be represented by a sequence of points, i.e., [ point _0, point _1, point _2,.. once, point _ { n-1} ], where n is a positive integer, and each point represents a coordinate in an image to be detected. arrow represents the direction in which the vehicle is allowed to travel and can be represented in a sequence of two points, namely point _ a, point _ B, pointed from point _ a to point _ B. labels represents the lane type of the current lane.

S102: and identifying at least partial image in the video to be detected, and determining first track information of the target vehicle in the video to be detected.

In this step, the target vehicle may include a vehicle belonging to a predetermined type, such as a large passenger car, a medium-sized, a heavy-duty truck, or the like. The image may include an image in which a vehicle driving picture exists. Each frame of image, or at least part of the image, in the video to be detected can be identified. At least part of the image may be an image obtained by extracting frames from the video to be detected. For example, the frame extraction may be performed randomly, or according to a set rule (for example, an equal interval frame extraction method), or the like, or the set rule may be changed in practical application, and the embodiment of the present disclosure is not limited specifically. The first track information may include a travel track of the target vehicle in the preset detection area, and specifically, may be coordinates corresponding to the travel track of the target vehicle in the preset detection area.

Determining a target vehicle in the video to be detected, in an optional implementation manner, extracting features of an image in the video to be detected, comparing the features with features of a vehicle of a preset type stored in advance, and determining the vehicle in the image as the target vehicle under the condition that the extracted features of the vehicle accord with the features of the vehicle of the preset type. In another optional implementation manner, feature extraction may be performed on an image in a video to be detected through a pre-trained network model capable of identifying a vehicle structural feature, the structural feature of the vehicle in the image is extracted, and the identified vehicle is determined to be the target vehicle when the structural feature of the vehicle indicates that the vehicle is a preset type of vehicle.

The method comprises the following specific steps of determining first track information of a target vehicle: firstly, the position of a target vehicle is identified, geometric center coordinates of the target vehicle can be found out and serve as coordinates corresponding to a running track point of the target vehicle at the current position, then, running track points of the target vehicle can be determined by identifying continuous frame images, or running track points of the target vehicle can be determined by identifying partial images obtained by frame skipping, and a plurality of running track points are connected into a running track according to time sequence information of the determined running track points.

S103: and determining the violation driving information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane.

In specific implementation, at least one passing lane through which the target vehicle passes is determined based on the first track information and the position information of each lane; and determining the violation driving information of the target vehicle based on the label information corresponding to each passing lane.

Here, given the position information of each lane, that is, the coordinates of the lane area, when the first trajectory information, that is, the coordinates corresponding to the travel track point is determined, each lane, that is, at least one passing lane, through which the target vehicle passes in the preset detection area may be determined by the coordinates corresponding to the travel track point. Then, the tag information corresponding to each passing lane may be determined, taking the same-direction three-row lane as an example, determining that the first trajectory information of the large truck includes that the large truck travels from the rightmost lane to the middle lane and then returns to the rightmost lane, and then determining that the passing lanes of the large truck passing through the preset detection area include the rightmost lane and the middle lane, and determining the tag information corresponding to the passing lane according to the tag information of each lane, that is, the rightmost lane corresponds to the legal traveling lane, and the middle lane corresponds to the overtaking lane.

The method comprises the steps that label information corresponding to each passing lane is utilized, whether a target vehicle runs on a violation driving lane or a overtaking lane can be accurately determined, whether the target vehicle runs in a violation driving lane or not can be accurately determined based on the label information and the first track information, and violation driving information of the target vehicle can be accurately determined.

In one embodiment, in the case where the target vehicle is determined to be in a second target lane based on the first track information, the violation traveling information is determined to include information that the target vehicle is traveling in a violation, wherein the second target lane is a violation traveling lane determined based on the tag information of each passing lane.

Whether the target vehicle runs in a second target lane can be accurately determined by using the label information, and the second target lane is a specific lane for judging illegal running, namely an illegal running lane; under the condition that the target vehicle runs in the second target lane, the illegal running of the target vehicle can be directly and accurately determined, namely the illegal running information can be more accurately determined.

In another embodiment, the network judges the running track of the target vehicle by identifying the geometric center position of the target vehicle, and when the network misjudges the geometric center position of the target vehicle, the running track of the target vehicle is deviated, and the target vehicle is possibly misjudged to be in a traffic violation running lane. Therefore, in order to prevent the network from misjudging the location of the target vehicle, in the case that the target vehicle is already determined to be in the second target lane, the driving track of the target vehicle on the second target lane may be further combined, and specifically, in the case that the driving distance of the target vehicle on the second target lane is determined to be greater than a preset distance (for example, 0.5m (meter)), it is determined that the target vehicle has illegal driving. Under the condition that the running distance of the target vehicle on the second target lane is determined to be smaller than or equal to the preset distance, whether the target vehicle returns to the original running lane or not can be further judged by combining the first track information, and if the target vehicle returns to the original running lane, the target vehicle is determined not to violate rules.

The driving distance may include a distance actually traveled by the target vehicle on the second target lane, or may be a straight length, i.e., a displacement amount, between the start point position and the end point position of the target vehicle in the first trajectory information.

Determining violation driving information of the target vehicle, and in one possible embodiment, determining second track information of the target vehicle in a first target lane based on first track information, wherein the first target lane is a overtaking lane determined based on label information of each passing lane; and then, determining the illegal driving information of the target vehicle based on the second track information.

Here, the second trajectory information may be partial trajectory information on the first target lane in the first trajectory information. The first target lane may be a specific lane for determining that the target vehicle is traveling violating the regulations. Reference may be made to fig. 3a, which is a schematic diagram illustrating a first trajectory information of a target vehicle on a three-row lane in the same direction. Alternatively, refer to fig. 3b, which is a schematic diagram illustrating another first trajectory information of the target vehicle on the equidirectional three-row lane. Where 31 denotes a target vehicle, 32 denotes first track information, 33 denotes second track information including a start point position 331 and an end point position 332 (respectively indicated by dashed circle boxes), 34 denotes a traffic lane, 35 denotes a first target lane (e.g., a passable lane), 36 denotes a second target lane (e.g., a violation traffic lane), and 37 denotes a third target lane (e.g., a legal traffic lane).

Here, the first target lane, the second target lane, and the third target lane may be preset lane types according to violation driving regulations. The first target lane and the second target lane can be specific lanes for judging that the target vehicle runs in a violation manner, the lane type of the first target lane can be set as a overtaking type, and the first target lane is the overtaking lane; the lane type of the second target lane can be set as a violation driving type, and the second target lane is a violation driving lane; the lane type of the third target lane may be set as a legal driving type, and the third target lane is a legal driving lane. Here, the preset lane type may be set according to an actual application scenario, and the embodiment of the present disclosure is not particularly limited.

And continuing the previous example, determining the label information corresponding to the passing lanes, namely that the rightmost lane corresponds to a legal driving lane, and the middle lane corresponds to a passable lane. And under the condition that the type of the target vehicle can be the overtaking type, the screened first target lane is an intermediate lane. Then, calculating second track information of the first track information in the middle lane, wherein in one case, as shown in fig. 3a, the second track information is the same as the first track information, which indicates that the large truck always runs on the middle lane; in another case, as shown in fig. 3b, if part of the first trajectory information is the same as the second trajectory information, it means that the truck travels on the other traffic lane first and then moves to the middle lane. The second trajectory information is the partial trajectory information in the middle lane (i.e., the first target lane) in the first trajectory information.

Then, determining the illegal driving information of the target vehicle based on the second track information, wherein in one case, if the driving distance of the target vehicle indicated by the second track information is less than a preset distance (such as 0.5m (meter)), the target vehicle can be determined not to have illegal driving, namely the illegal driving information does not include the illegal driving information of the target vehicle; the driving distance may include a distance actually traveled by the target vehicle on the lane, or may be a straight length between the start point position and the end point position of the target vehicle in the second trajectory information, that is, a displacement amount. In another case, if the driving distance of the target vehicle indicated by the second track information is greater than or equal to the preset distance, it can be determined that the target vehicle has overtaking willingness, at this time, it can be further determined whether the target vehicle has overtaking behavior, if the overtaking behavior exists, it can be determined that the target vehicle does not have violation driving, that is, the violation driving information does not include the information that the target vehicle runs in violation driving; on the contrary, if the overtaking behavior does not exist, the target vehicle can be determined to have the illegal driving because the target vehicle runs on the overtaking lane, namely the illegal driving information comprises the illegal driving information of the target vehicle. The preset displacement amount may be set according to a specific application scenario, and the embodiment of the present disclosure is not specifically limited. Here, the case where the passing behavior exists includes that the trajectory between the first trajectory information and the second trajectory information has overlapping information on the extension line in the lane direction, that is, the target overlapping information described below is not empty.

The method comprises the steps that label information is fully utilized, whether a target vehicle runs in a first target lane can be determined accurately, and the first target lane is a specific lane for judging illegal driving, for example, the first target lane is a passable lane; under the condition that the target vehicle runs in the first target lane, the running track of the target vehicle in the first target lane, namely the second track information, can be combined to accurately determine whether the target vehicle has overtaking behaviors, and the violation running information can be accurately determined based on the overtaking behaviors.

In some embodiments, whether the target vehicle has a violation behavior is judged, and when the violation behavior is specifically implemented, first, third trajectory information corresponding to other vehicles on a third target lane in a time period corresponding to the second trajectory information may be obtained, where the third target lane is a legal driving lane determined based on the label information of each passing lane; finally, determining the violation driving information of the target vehicle based on the target overlapping information between the tracks respectively corresponding to the second track information and the third track information; the target overlap information includes overlap information on a lane direction extension line. The embodiment can accurately judge the behavior information of the current target vehicle relative to other vehicles, such as overtaking behaviors and the like, and further can obtain accurate violation driving information of the target vehicle.

Here, in the case where it has been determined that the target vehicle is on the first target lane, the third trajectory information corresponding to the other vehicle located in the third target lane, that is, the legal travel lane, may be directly acquired according to the tag information in the time period corresponding to the second trajectory information. In some embodiments, third trajectory information of other vehicles on a third target lane may be acquired upon detecting that the target vehicle is just located on the first target lane.

In other embodiments, third trajectory information of the other vehicle in a third target lane may be acquired after the target vehicle is detected to travel on the first target lane for a certain distance. Specifically, first displacement information of the target vehicle within the first target lane may be determined based on the second trajectory information; and under the condition that the first displacement information meets the preset displacement condition, acquiring third track information corresponding to other vehicles on a third target lane in a time period corresponding to the second track information.

In an actual application process, how to acquire the third trajectory information of the other vehicle may be set according to local illegal driving regulations, and the embodiment of the disclosure is not particularly limited. Wherein the first displacement information may include a length of a straight line between the start point position and the end point position of the target vehicle in the second trajectory information, i.e., a displacement amount. The preset displacement condition may be set according to a specific application scenario, for example, the preset displacement condition may be that the displacement indicated by the first displacement information is greater than 10 meters (m), and the embodiment of the present disclosure is not specifically limited.

The time period corresponding to the second track information may be a time period from the start position corresponding to the second track information to the end position corresponding to the second track information. Other vehicles in the third target lane during the time period may be considered vehicles that the target vehicle is preparing to pass. And calculating third track information corresponding to other vehicles, wherein the calculation mode can refer to the calculation mode for calculating the first track information, and repeated parts are not described herein again.

Second displacement information of the other vehicle within the third target lane is determined based on the third trajectory information, and the second displacement information may include a length of a straight line between the start point position and the end point position of the target vehicle in the third trajectory information, that is, a displacement amount. And then, determining the illegal driving information of the target vehicle according to the displacement corresponding to the first displacement information and the displacement corresponding to the second displacement information.

In addition, in the above-described scenario of determining whether the target vehicle has a violation, in one case, the target vehicle starts to overtake from the legal travel lane to the overtake lane, and then returns to the original legal travel lane. In another case, the target vehicle may start to overtake from the legal driving lane to the overtake lane, and then may not return to the original legal driving lane in time. The target vehicle in both cases may have a violation.

In some embodiments, if the displacement amount corresponding to the first displacement information is larger than the displacement amount corresponding to the second displacement information (it is determined that overtaking behavior exists), and the displacement amount corresponding to the first displacement information is also larger than the first preset displacement amount, wherein the first preset displacement amount is larger than the displacement amount corresponding to the second displacement information, it may be determined that the target vehicle runs in violation, that is, the violation running information includes information that the target vehicle runs in violation. The embodiment can include the two situations 1, even if the target lane exceeds other vehicles on the overtaking lane and then returns to the original legal driving lane, the target vehicle also has violation behaviors because the target vehicle has too long driving path on the overtaking lane. 2. The target lane surpasses other vehicles on the overtaking lane, then the original legal driving lane is not returned, and under the condition that the target lane continuously drives on the overtaking lane, the displacement corresponding to the first displacement information is larger than the first preset displacement, so that the target vehicle also has violation behaviors.

For example, taking the same-direction three-row lane as an example, it is known that the displacement amount corresponding to the first displacement information is 300m, the displacement amount corresponding to the second displacement information is 120m, the first preset displacement amount is set to be 160m, and since the first displacement information is greater than the second displacement information, it can be determined that the truck overtakes the middle lane, but since the first displacement information is greater than the first preset displacement amount, that is, the overtaking distance is too long, the target vehicle runs in violation of the traffic.

In other embodiments, if the displacement amount corresponding to the first displacement information is greater than the displacement amount corresponding to the second displacement information (it is determined that the overtaking behavior exists), and the displacement amount corresponding to the first displacement information is smaller than a second preset displacement amount, where the second preset displacement amount is greater than the displacement amount corresponding to the second displacement information, it may be determined that the violation driving information does not include the information that the target vehicle runs in violation.

For example, taking the same-direction three-row lane as an example, it is known that the displacement amount corresponding to the first displacement information is 150m, the displacement amount corresponding to the second displacement information is 120m, and the second preset displacement amount is set to be 160m, and since the first displacement information is greater than the second displacement information, it can be determined that the large truck overtakes the middle lane, and in addition, since the first displacement information is less than the second preset displacement amount and does not travel on the overtaking lane for a long distance, it can be determined that the information of the large truck during illegal driving is not included in the information of the large truck.

It should be noted that the first preset displacement and the second preset displacement may be set according to an actual application scenario, and the embodiment of the present disclosure is not specifically limited.

The target overlap information includes overlap information on a lane direction extension line, the lane direction being the lane direction 233 indicated in fig. 2, the overlap information including overlap information in which trajectories corresponding to the first trajectory information and the second trajectory information, respectively, are projected on the lane extension line.

In order to improve the accuracy of the calculated distance that the target vehicle actually exceeds the other vehicles, the distance that the target vehicle actually exceeds the other vehicles may be calculated by using projection information of the respective travel track information. In some embodiments, first projection information of the second trajectory information on the lane direction extension and second projection information of the third trajectory information on the lane direction extension are determined; determining target overlap information between the first projection information and the second projection information; and determining the illegal driving information of the target vehicle based on the target overlapping information and the first projection information.

The projection information (e.g., the first projection information and the second projection information) may include a scale, i.e., a ratio of a projection length to a projection length on the map, a position (coordinate) on the map, and the like.

The first projection information may be a projection of the second trajectory on the lane direction extension line, which is determined based on the trajectory coordinates in the second trajectory information. The second projection information may be a projection of the third trajectory on the lane direction extension line, which is determined based on the trajectory coordinates in the third trajectory information.

Reference may be made to fig. 4a, which is a schematic diagram illustrating a target overlapping information. Alternatively, see fig. 4b, which is a schematic diagram illustrating another target overlapping information. Wherein 40 represents a lane direction extension line, 41 represents second track information corresponding to the target vehicle, including a starting point position 411 and an ending point position 412 (respectively represented by dashed circle boxes), and 413 represents a projection length corresponding to the first projection information; 42 denotes a overtaking available lane; 43 denotes a legal driving lane; 44 denotes a running plane of the target vehicle; 45 denotes a lane line, i.e., a critical line between two adjacent lanes; 46 denotes third track information, 461 denotes a projection length corresponding to the second projection information; 47 denotes a projection length corresponding to target overlap information between the first projection information and the second projection information; 48 denotes a projection length corresponding to the projection information remaining after the first projection information is removed from the target superimposition information (including a sum of projection lengths corresponding to at least one piece of projection information remaining after the first projection information is removed from the target superimposition information, for example, a sum of the projection length denoted by 481 and the projection length denoted by 482 in fig. 4 b).

And determining the illegal driving information of the target vehicle based on the target overlapping information and the first projection information, wherein in some embodiments, referring to fig. 4a, in the case that the target overlapping information is located at one end of the first projection information, and the projection length corresponding to the projection information left after the target overlapping information is removed from the first projection information is greater than a first preset length, the illegal driving information is determined to include the illegal driving information of the target vehicle.

Specifically, one end of the projection length corresponding to the target overlapping information is located at one end of the projection length corresponding to the first projection information, and if the remaining projection length (i.e., the projection length indicated by 48) is greater than the first preset length after the projection length corresponding to the first projection information is removed from the projection length corresponding to the target overlapping information, it may be determined that the target vehicle is in violation of driving, that is, the violation of driving information includes information that the target vehicle is in violation of driving.

In other embodiments, referring to fig. 4b, in the case that the target overlapping information is not located at any end of the first projection information, that is, at the middle position of the first projection information, and the projection length corresponding to the projection information remaining after the first projection information is removed from the target overlapping information is greater than the first preset length, it is determined that the violation driving information includes information that the target vehicle runs in violation.

Specifically, any end of the projection length corresponding to the target overlapping information is not located at any end of the projection length corresponding to the first projection information, and if the remaining projection length (including the sum of the projection length represented by 481 and the projection length represented by 482) is greater than the first preset length after the projection length corresponding to the first projection information is removed from the projection length corresponding to the target overlapping information, it can be determined that the target vehicle is in violation of driving, that is, the violation of driving information includes information that the target vehicle is in violation of driving.

In other embodiments, referring to fig. 4b, in the case that the target overlapping information is not located at any end of the first projection information, that is, at the middle position of the first projection information, and in the projection information remaining after the first projection information excludes the target overlapping information, the projection length corresponding to part of the projection information, for example, the projection length indicated by 482, is greater than the first preset length, it is determined that the violation driving information includes information that the target vehicle runs in violation.

The first preset length may be defined according to an actual application scenario, and the embodiment of the present disclosure is not specifically defined.

In some embodiments, in the case where the target overlap information is not empty, determining that there is overtaking behavior for the target vehicle; and under the condition that the target vehicle has overtaking behavior and the projection length corresponding to the first projection information is smaller than the second preset length, determining that the illegal driving information does not include the illegal driving information of the target vehicle.

The target overlapping information is not empty, namely the track between the first track information and the second track information is overlapped on the extension line of the lane direction, and the overtaking behavior of the target vehicle can be determined according to the overtaking behavior definition; at this time, if the projection length corresponding to the first projection information is smaller than the second preset length, it can be determined that the target vehicle has overtaking behavior and does not violate rules; if the projection length corresponding to the first projection information is greater than or equal to the second preset length, it can be determined that the target vehicle has a behavior of continuously driving on the overtaking lane under the condition that the overtaking behavior exists, and therefore the target vehicle can be determined to be in violation of driving.

The target vehicle executes the overtaking behavior in the overtaking lane, and normally travels 150m, so that overtaking can be realized, wherein a second preset length, such as 150m, can be set according to the actual application scene. Under the condition that the overtaking behavior exists in the target vehicle and the overtaking travel track does not exceed 150m, the illegal travel information can be determined not to include the illegal travel information of the target vehicle. The specific value of the second preset length may also be set according to an actual application scenario, and the embodiment of the present disclosure is not specifically limited.

For step S102, determining first track information of the target vehicle, and in some embodiments, further identifying at least a partial image in the video to be detected, and determining a key point corresponding to the target vehicle in the at least partial image; and then, determining first track information of the target vehicle in the video to be detected based on the identified key points corresponding to the target vehicle.

Specifically, the key point may include a wheel position of the target vehicle, and then, the closed polygon determined by the wheel position of the target vehicle represents the overall position of the target vehicle, for example, an overlapping area of the closed polygon determined by the wheel position and the lane area may be determined, in a case that the overlapping area is greater than a preset area threshold, the current position of the target vehicle may be determined as the lane area, and then, a geometric center point of the closed polygon represents a travel track point of the current target vehicle at the current position of the lane area. And then, connecting a plurality of driving track points into a driving track according to the determined time sequence information of the driving track points, namely determining first track information of the target vehicle.

Here, the preset area threshold may be set to 50%. Alternatively, the setting may also be performed according to a specific application scenario, and the embodiment of the present disclosure is not specifically limited.

In some embodiments, after determining chapter travel information, a travel report may also be generated as needed for the mission. Specifically, the violation running report of the target vehicle may be generated in the case where it is determined that the target vehicle runs in violation.

The violation travel report may include, but is not limited to, at least one of: the method comprises the following steps that violation driving information of a target vehicle, structural information of the target vehicle, a starting frame image corresponding to first track information, an ending frame image corresponding to the first track information, a middle frame image corresponding to the first track information and a target frame image corresponding to the first track information are obtained; the target frame image comprises a target vehicle meeting preset conditions.

The violation driving information of the target vehicle is the violation driving information of the target vehicle determined in the processes from S101 to S103; the structured information of the target vehicle may include, but is not limited to, a type, a color, a track, a wheel position, a license plate type, a license plate color, a license plate number, etc. of the target vehicle. The structured information of the target vehicle may be set according to task needs, and the embodiment of the present disclosure is not particularly limited.

The start frame image corresponding to the first track information may be an image of the target vehicle in the video to be detected, where the acquired first frame includes the image of the target vehicle. The ending frame image corresponding to the first track information may be an image in which the last frame acquired in the video to be detected contains the target vehicle. The first track information corresponds to any frame containing the image of the target vehicle in the time period between the time corresponding to the starting position point and the time corresponding to the ending position point of the first track information in the video to be detected. The target frame image corresponding to the first track information may be an image which is shot in the video to be detected and contains the clearest structure of the target vehicle, for example, an image which is not close to the edge of the video to be detected, or an image which is clear in the outline of the target vehicle. Here, the preset condition may include that the target vehicle structure is clear. Alternatively, the setting may be performed according to an actual application scenario, and the embodiment of the present disclosure is not particularly limited.

By using the violation running report recorded with more detailed running information, the violation running behavior of the target vehicle can be comprehensively and accurately characterized, related personnel are informed to stop the violation behavior in time, and the risk of traffic accidents is reduced.

Reference may be made to fig. 5, which is a schematic illustration of a violation travel report. The method comprises the following steps: reference numeral 51 denotes the violation traveling information, 52 denotes the structural information of the target vehicle, 53 denotes the start frame image, 54 denotes the end frame image, 55 denotes the intermediate frame image, and 56 denotes the target frame image.

Through the steps S101 to S103, the images are identified by using the video structured platform and the like, the first track information of the target vehicle is determined based on the structural information of the target vehicle obtained through identification, then the corresponding marking information (used for judging the illegal lane occupation condition) of the lane marked in the preset detection area is combined, the behavior of the target vehicle is analyzed, whether the condition of illegal driving of the target vehicle exists or not is judged, namely the illegal driving information of the target vehicle, the capability of the existing platforms such as the video structured platform and the like is fully utilized, the structural detection is carried out on the target vehicle, the detection cost of illegal driving of the vehicle is reduced, meanwhile, the behavior detection is carried out on the target vehicle by combining the identification result, the detection precision and the detection efficiency can be improved, and the human resources can be saved.

Illustratively, referring to fig. 6, it is a schematic diagram of a decision flow of detection of violation of driving of the target vehicle, including S601-S606:

s601: and identifying at least partial image in the video to be detected, and determining the structural information of the target vehicle.

In specific implementation, vehicle identification is carried out on images in a video to be detected, vehicle images including a target vehicle are determined, and vehicle information corresponding to the target vehicle is determined based on the vehicle images; the method comprises the steps of performing license plate recognition on a vehicle image, determining a license plate image comprising a license plate, and determining license plate information corresponding to the license plate based on the license plate image; and determining the structural information corresponding to the target vehicle based on the vehicle information and the license plate information.

Here, the vehicle detection model may be used to perform vehicle recognition on the image in the video to be detected, determine a vehicle image containing the target vehicle, and perform model processing on the vehicle image to obtain vehicle information, which may include but is not limited to: vehicle type, vehicle color, wheel position information, vehicle trajectory information, such as first trajectory information of the target vehicle, and the like.

Then, the license plate detection model can be used for carrying out license plate recognition on the vehicle image, the license plate position in the target vehicle is determined, the license plate image can be intercepted from the vehicle image, then, license plate structural recognition is carried out on the license plate image, and license plate information is determined, wherein the license plate information can include but is not limited to: license plate type, license plate color, license plate number, etc.

The network structure in the vehicle detection model or the license plate detection model may include a two-stage detection network structure (fast RCNN, etc.) and a single-stage detection network structure (RetinaNet, etc.).

The video to be detected is detected by fully utilizing the existing platforms such as the video structured platform, the structure of the target vehicle in the video is analyzed, the structured information corresponding to the target vehicle is comprehensively reflected, and the detection cost of vehicle illegal running can be reduced.

S602: judging whether the target vehicle runs in the second target lane, if so, executing S605, otherwise, executing S603;

s603: judging whether first displacement information of the target vehicle in the first target lane meets a preset displacement condition, if so, executing S604, and if not, executing S606;

s604: judging whether the target vehicle has overtaking behaviors, if so, executing S606; if not, S605 is executed.

S605: generating the violation running information and ending the process;

s606: the target vehicle does not run violating the regulations.

For S604, for example, see fig. 7, which is a schematic flow chart illustrating the determination of whether the target vehicle has the overtaking behavior, including S701 to S709:

s701: determining first projection information of second track information of the target vehicle on a lane direction extension line;

s702: acquiring the running track information of the vehicles on other lanes in a time period corresponding to the second track information, and determining third projection information of the running track information on a lane direction extension line;

s703: judging whether overlapping information exists between the first projection information and the third projection information; if so, executing S704; otherwise, S708 is executed;

s704: judging whether track points on a third target lane exist in the running track information or not; if so, then S705 is executed: otherwise, S708 is executed;

s705: determining third track information corresponding to other vehicles on a third target lane, and determining second projection information of the third track information on a lane direction extension line;

s706: judging whether target overlapping information exists between the first projection information and the second projection information, if so, executing S707; otherwise, go to S708;

s707: and determining that the target vehicle has overtaking behaviors, and ending the process.

S708: judging whether to acquire the next vehicle on the other lane, if so, executing S702 in a circulating manner, and if not, executing S709;

s709: and determining that the target vehicle has no overtaking behavior.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same inventive concept, the embodiment of the present disclosure further provides a vehicle illegal running detection device corresponding to the vehicle illegal running detection method, and as the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the vehicle illegal running detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated parts are not described again.

Referring to fig. 8, a schematic diagram of a vehicle illegal driving detection device provided in the embodiment of the present disclosure is shown, where the device includes: an information acquisition module 801, a track determination module 802 and a violation detection module 803; wherein the content of the first and second substances,

the information acquisition module 801 is configured to acquire a to-be-detected video corresponding to a preset detection area and label information corresponding to each lane in the preset detection area;

a track determining module 802, configured to identify at least part of images in the video to be detected, and determine first track information of a target vehicle in the video to be detected;

and the violation detection module 803 is configured to determine violation driving information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane.

the violation detection module 803 is configured to determine, based on the first track information and the position information of each lane, at least one passing lane through which the target vehicle passes;

and determining the violation driving information of the target vehicle based on the label information corresponding to each passing lane and the first track information.

In an optional embodiment, the violation detection module 803 is configured to determine second track information of the target vehicle in a first target lane based on the first track information, where the first target lane is a vehicle-overtaking lane determined based on the tag information of each passing lane;

In an optional embodiment, the violation detection module 803 is configured to determine that the violation driving information includes information that the target vehicle runs in a violation manner if it is determined that the target vehicle is in a second target lane based on the first track information, where the second target lane is a violation driving lane determined based on the tag information of each passing lane.

In an optional embodiment, the labeling information further includes a lane direction; the violation detection module 803 is configured to obtain third trajectory information corresponding to other vehicles on a third target lane in the time period corresponding to the second trajectory information, where the third target lane is a legal driving lane determined based on the label information of each passing lane;

In an optional embodiment, the violation detection module 803 is specifically configured to determine, based on the second track information, first displacement information of the target vehicle in the first target lane;

In an optional embodiment, the violation detection module 803 is further configured to determine, before determining the violation traveling information of the target vehicle based on the target overlapping information between the tracks corresponding to the second track information and the third track information, first projection information of the second track information on a lane direction extension line, and second projection information of the third track information on the lane direction extension line;

the violation detection module 803 is configured to determine violation driving information of the target vehicle based on the target overlapping information and the first projection information.

In an optional implementation manner, the violation detection module 803 is configured to determine that the violation driving information includes information that the target vehicle runs in violation, when the target overlapping information is located at one end of the first projection information, and a projection length corresponding to projection information remaining after the first projection information is removed from the target overlapping information is greater than a first preset length.

In an optional embodiment, the violation detection module 803 is configured to determine that there is overtaking behavior of the target vehicle if the target overlapping information is not empty;

In an optional implementation manner, the track determining module 802 is configured to identify at least a partial image in the video to be detected, and determine a keypoint corresponding to the target vehicle in the at least a partial image;

In an optional embodiment, the device further comprises a report generating module 804, which is used for generating a violation running report of the target vehicle under the condition that the violation running information of the target vehicle is determined;

In an optional implementation manner, the apparatus further includes an information determining module 805, configured to perform vehicle identification on the images in the video to be detected, determine a vehicle image including the target vehicle, and determine vehicle information corresponding to the target vehicle based on the vehicle image;

The description of the processing flow of each module in the vehicle illegal running detection device and the interaction flow among the modules can refer to the related description in the above vehicle illegal running detection method embodiment, and the detailed description is omitted here.

Based on the same technical concept, the embodiment of the application also provides computer equipment. Referring to fig. 9, a schematic structural diagram of a computer device provided in an embodiment of the present application includes:

a processor 91, a memory 92, and a bus 93. Wherein the memory 92 stores machine-readable instructions executable by the processor 91, the processor 91 is configured to execute the machine-readable instructions stored in the memory 92, and when the machine-readable instructions are executed by the processor 91, the processor 91 performs the following steps: s101: acquiring a to-be-detected video corresponding to a preset detection area and marking information corresponding to each lane in the preset detection area; s102: identifying at least partial image in a video to be detected, and determining first track information of a target vehicle in the video to be detected; s103: and determining the violation driving information of the target vehicle based on the first track information of the target vehicle and the marking information corresponding to each lane.

The memory 92 includes a memory 921 and an external memory 922; the memory 921 is also referred to as an internal memory, and is used for temporarily storing the operation data in the processor 91 and data exchanged with an external memory 922 such as a hard disk, the processor 91 exchanges data with the external memory 922 through the memory 921, and when the computer device is operated, the processor 91 communicates with the memory 92 through the bus 93, so that the processor 91 executes the execution instructions mentioned in the above method embodiments.

The disclosed embodiment also provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle illegal running detection method in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The disclosed embodiment also provides a computer program product, which comprises computer instructions, and the computer instructions are executed by a processor to realize the steps of the vehicle illegal driving detection method. Where the computer program product may be any product capable of implementing the vehicle violation detection method described above, some or all aspects of the computer program product that contribute to the prior art may be embodied in a Software product (e.g., Software Development Kit (SDK)) that may be stored on a storage medium and that may include computer instructions for causing an associated device or processor to perform some or all of the steps of the vehicle violation detection method described above.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and is not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the modules is only one logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional modules in the embodiments of the present disclosure may be integrated into one processing module, or each module may exist alone physically, or two or more modules are integrated into one module.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A vehicle illegal driving detection method is characterized by comprising the following steps:

2. The method according to claim 1, wherein the labeling information includes position information of the corresponding lane and tag information of the corresponding lane;

3. The method of claim 2, wherein the determining of the violation driving information of the target vehicle based on the tag information corresponding to each passing lane comprises:

4. The method of claim 2, wherein the determining of the violation driving information of the target vehicle based on the tag information corresponding to each passing lane comprises:

5. The method of claim 3, wherein the annotation information further comprises a lane direction; the determining of the illegal driving information of the target vehicle based on the second track information comprises the following steps:

6. The method according to claim 5, wherein the obtaining third trajectory information corresponding to other vehicles in a third target lane during the time period corresponding to the second trajectory information comprises:

7. The method according to claim 5 or 6, wherein before the determining of the violation driving information of the target vehicle based on the target overlap information between the tracks corresponding to the second track information and the third track information, respectively, further comprises:

8. The method of claim 7, wherein the determining violation travel information for the target vehicle based on the target overlap information and the first projection information comprises:

9. The method of claim 7, wherein the determining violation travel information for the target vehicle based on the target overlap information and the first projection information comprises:

10. The method according to claim 1, wherein the identifying at least a portion of the image in the video to be detected and determining the first trajectory information of the target vehicle in the video to be detected comprises:

11. The method of any one of claims 1 to 9, further comprising, after determining the violation travel information:

12. The method of claim 11, further comprising the step of determining the structured information of the target vehicle:

13. A vehicle violation travel detection device comprising:

14. A computer device, comprising: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating over the bus when a computer device is operated, the machine readable instructions when executed by the processor performing the steps of the vehicle violation detection method as recited in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, executes the steps of the vehicle violation running detection method according to any one of claims 1 to 12.