WO2020087743A1

WO2020087743A1 - Non-motor vehicle traffic violation supervision method and apparatus and electronic device

Info

Publication number: WO2020087743A1
Application number: PCT/CN2018/124833
Authority: WO
Inventors: 郑文先; 陈耀沃; 邹博
Original assignee: 深圳云天励飞技术有限公司
Priority date: 2018-11-01
Filing date: 2018-12-28
Publication date: 2020-05-07
Also published as: CN109635645A

Abstract

A non-motor vehicle traffic violation supervision method and apparatus and an electronic device. The method comprises: obtaining image information (101); performing visual tracking on the non-motor vehicle features and the face features to obtain motion information of the non-motor vehicle and motion information of the personnel (102); matching a driver and a passenger for the non-motor vehicle according to the motion information of the non-motor vehicle and the motion information of the personnel (103); determining whether the non-motor vehicle triggers a preset illegal condition or not according to the motion information of the non-motor vehicle (104); and if the non-motor vehicle has the illegal behavior, confirming identity information of the driver and the passenger of the illegal non-motor vehicle (105). Due to the fact that the face features are matched with the non-motor vehicle features, drivers and passengers of the non-motor vehicles are matched, and traffic violation behaviors of the non-motor vehicles can be supervised more effectively.

Description

Non-motor vehicle traffic illegal supervision method, device and electronic equipment

This application requires priority to be submitted to the Chinese Patent Office on November 1, 2018, with the application number 201811296616.2 and the invention titled "Methods, Devices and Electronic Equipment for Non-Motor Vehicle Traffic Illegal Supervision", the entire content of which is cited by reference Incorporated in this application.

Technical field

The invention relates to the technical field of image recognition, and in particular to a method, device and electronic equipment for non-motor vehicle traffic violation supervision.

Background technique

Image recognition is one of the current technologies commonly used in traffic management, for example: detection of traffic violation events based on license plate image recognition or detection of other traffic events based on image recognition of other vehicle features. At present, image recognition in traffic management mainly focuses on the identification of motor vehicle license plates and vehicle characteristics, and determines illegal drivers from license plate and vehicle information to supervise traffic violations. However, for non-motor vehicles that have not been licensed in accordance with traffic management requirements, it is currently not possible to supervise the traffic violations of non-motor vehicles.

Summary of the invention

Embodiments of the present invention provide a method and device for monitoring non-motor vehicle traffic violations, electronic equipment, and computer-readable storage media, which can more effectively supervise non-motor vehicle traffic violations.

In a first aspect, an embodiment of the present invention provides a non-motor vehicle traffic violation supervision method, including:

Obtain image information, the image information includes non-motor vehicle features of non-motor vehicles and human face features;

Visually tracking the characteristics of the non-motor vehicle and the facial features to obtain the movement information of the non-motor vehicle and the movement information of the person;

Matching driver and passengers for the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel;

According to the motion information of the non-motor vehicle, determine whether the non-motor vehicle triggers a preset illegal condition;

If the non-motor vehicle triggers the preset illegal condition, the identity information of the driver of the illegal non-motor vehicle is confirmed.

Optionally, after confirming the identity information of the driver of the illegal non-motor vehicle, the method further includes:

Generate illegal information according to the illegal conditions triggered by the non-motor vehicle;

Sending the illegal information to a prompting device, where the prompting device is used to remind the driver of the illegal non-motor vehicle.

Optionally, the motion information of the non-motor vehicle includes motion information of the characteristics of the non-motor vehicle, and the motion information of the person includes motion information of the facial characteristics;

The matching the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel includes:

Acquiring facial features within the preset range of the non-motor vehicle features, and matching the non-locomotive driver with the non-locomotive based on the motion information of the non-motor vehicle features and the facial feature motion information .

Optionally, the motion information according to the non-motor vehicle characteristics and the facial feature motion information in the preset range for the non-locomotive matching driver and passengers includes:

Forming a face set according to the motion information of the non-motor vehicle features and the motion information of the face features within the preset range;

Match the driver and passenger to the non-locomotive according to the face set.

Optionally, the motion information of the non-motor vehicle feature includes the speed of the non-motor vehicle feature, and the motion information of the face feature includes the speed of the face feature;

The forming of a face set according to the movement information of the non-motor vehicle characteristics and the movement information of the face characteristics within the preset range includes:

Comparing the speed of the non-motor vehicle feature with the speed of the face feature to obtain a speed comparison result;

According to the speed comparison result, the face set is formed.

Optionally, matching the non-locomotive occupants according to the face set includes:

Detecting the number of face features in the face set to determine whether there are multiple face features in the face set;

If there are multiple facial features, the facial features closest to the non-motor vehicle feature pixels are selected from the multiple facial features to determine the facial features of the driver and occupant to obtain the driving characteristics of the non-motor vehicle Passengers.

Optionally, the confirmation of the identity information of the driver of the illegal non-motor vehicle includes:

From the acquired image information, select an image with the facial features of the driver of the illegal non-motor vehicle for quality evaluation, and obtain an image quality score;

According to the image quality score, select the image with the highest image quality score;

Extract the facial features of the driver of the illegal non-motor vehicle from the image with the highest image quality score;

The facial features of the driver of the illegal non-motor vehicle are compared with the background identity image library to determine the identity information of the driver.

In a second aspect, an embodiment of the present invention provides a non-motor vehicle traffic violation monitoring device, including:

The first acquisition module is used to acquire image information, the image information includes non-motor vehicle features of the non-motor vehicle and human face features of the person;

A second acquisition module, configured to visually track the characteristics of the non-motor vehicle and the facial features, and obtain the motion information of the non-motor vehicle and the motion information of the person;

A matching module, configured to match the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel;

The judging module is used for judging whether the non-motor vehicle triggers a preset illegal condition according to the motion information of the non-motor vehicle;

The confirmation module is used to confirm the identity information of the driver of the illegal non-motorized vehicle if the illegal behavior of the non-motorized vehicle exists.

In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, when the processor executes the computer program The steps in the non-motor vehicle traffic law supervision method provided by the embodiments of the present invention are implemented.

According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium that stores a computer program on the computer-readable storage medium, and the computer program is executed by a processor to implement the non-motor vehicle traffic provided by the embodiment of the present invention. Steps in illegal supervision methods.

In the embodiment of the present invention, image information is obtained, and the image information includes non-motor vehicle features of a non-motor vehicle and human face features; visually tracking the non-motor vehicle features and the face features to obtain the Non-motor vehicle movement information and the person's movement information; according to the non-motor vehicle movement information and the person's movement information for the non-locomotive matching driver and passenger; based on the non-motor vehicle movement information, It is determined whether the non-motor vehicle triggers a pre-set illegal condition; if the non-motor vehicle has illegal behavior, the identity information of the driver of the illegal non-motor vehicle is confirmed. Due to the matching of face features and non-motor vehicle features, so as to match the drivers of non-motor vehicles, it is possible to more effectively supervise the traffic violations of non-motor vehicles.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, without paying any creative work, other drawings can be obtained based on these drawings.

FIG. 1 is a schematic flowchart of a method for monitoring non-motor vehicle traffic violations provided by an embodiment of the present invention;

2 is a schematic flowchart of another non-motor vehicle traffic violation supervision method provided by an embodiment of the present invention;

3 is a schematic flowchart of another non-motor vehicle traffic violation supervision method provided by an embodiment of the present invention;

4 is a schematic flowchart of another non-motor vehicle traffic violation supervision method provided by an embodiment of the present invention;

5 is a schematic structural diagram of a non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention;

6 is a schematic structural diagram of another non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention;

7 is a schematic structural diagram of another non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention;

8 is a schematic structural diagram of another non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention;

9 is a schematic structural diagram of another non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention;

10 is a schematic structural diagram of another non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention;

11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a method for non-motor vehicle traffic violation supervision provided by an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps:

101. Acquire image information, where the image information includes non-motor vehicle features of a non-motor vehicle and human face features of a person.

Wherein, the above image information can be collected by traffic cameras installed at intersections or roadsides. The traffic cameras used for collection can be shared with motor vehicle cameras and / or pedestrian cameras, or can be separately provided non-motor vehicles Capture camera. The above-mentioned non-motorized vehicle may be a non-motorized vehicle such as a bicycle, a tricycle, an electric vehicle, a balance bike, etc. The above-mentioned non-motorized vehicle characteristic may be a non-motorized vehicle characteristic such as a head, a handle, a pedal, a tire, and the like. The aforementioned facial features may be facial features such as facial contours, mouth, eyes, nose, etc.

It should be noted that the image information extracted in step 101 may include one or more non-motor vehicle features and one or more face features. In addition, the above image information may also be referred to as picture information, visual information, or picture frame information.

In some possible embodiments, the image information further includes the posture features of the person. The posture features may be posture features such as hand posture, foot posture, upper body posture, etc. By analyzing the posture features of the person, it can be determined whether the person has irregularities. In the case of driving a non-motorized vehicle, for example, it is possible to determine whether the person's hand is away from the handle, whether the person's foot is placed at a designated position, etc., through posture characteristics.

102. Track the characteristics of the non-motor vehicle and the facial features to obtain the motion information of the non-motor vehicle and the motion information of the person.

In this step, tracking refers to continuously acquiring images of non-motor vehicle features and human faces. After acquiring non-motor vehicle features and human face features, the non-motor vehicle features and human face features can be adjusted by adjusting the shooting angle of the camera. For tracking, for example, two sets of cameras can be set for shooting, one set of cameras is used for shooting still images, and one set of cameras is used for tracking and shooting dynamic images. Of course, the two sets of cameras are only for the convenience of explaining the way of acquiring images, and are not for implementing the present invention For example, it may also be one or more groups of cameras, and the number of each group of cameras is at least one. Of course, in some possible embodiments, the camera may also use a wide-angle camera, which has a wide field of view, so that You can track non-motor vehicle features and face features without adjusting the camera's shooting angle. The motion information of the non-motor vehicle features can be calculated through the displacement and time of the non-motor vehicle features on the multi-frame image. The motion information of the non-motor vehicle features is the motion information of the motor vehicle. For example, the first frame image is related to the first The interval between 200 frames of images is 4s, the position of the non-motorized vehicle in the first frame of the image is assumed to be 0m, and the position of the 200th frame of the image is 20m, then the average speed of the non-motorized vehicle is 5m / s, the above unit It can be a meter or a pixel on an image, which is not specifically limited here. The motion information of the non-motor vehicle may be speed information or trajectory information of the non-motor vehicle during driving. Similarly, the motion information of the person may be speed information or trajectory information of the person. Within the field of view of the camera, the motion information of the non-motor vehicle can be divided according to the relative motion state of the non-motor vehicle and the camera, and the relative motion state can include: forward driving, forward driving, stop driving, etc. The non-motorized vehicle is driving toward the camera. In this state, the camera can collect the facial features of the non-motorized person, and the facial features obtained in this state are easy to identify. The corresponding motion information can be the coming speed , And other trajectory information. Going forward refers to the driving of the non-motorized vehicle facing away from the camera. The corresponding motion information can be the moving information such as going speed and going track. Stopping refers to the stop of the non-motorized vehicle. The corresponding motion information can be the stopping speed and stopping. Movement information such as trajectory, of course, in the stop state, the stop speed is zero and the stop trajectory is a point.

It should be noted that the field of view of the camera may also be referred to as the field of view of the camera.

In some possible embodiments, the non-motor vehicle features appearing in the video may be tracked by the Kalman filter algorithm, so as to track the non-motor vehicle and obtain the motion information of the non-motor vehicle.

103. Match the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the person.

In this step, the motion information of the non-motorized vehicle may be speed information or trajectory information of the non-motorized vehicle during driving. Similarly, the motion information of the person may be the speed information or trajectory information of the person. The similarity of the motion information between the non-motor vehicle and the driver can be used to match the corresponding driver to the non-motor vehicle. For example, the non-motor vehicle with the same speed can be matched with the personnel to obtain the driver, or can Match the person with the speed closest to the speed of the non-motor vehicle to the driver, or you can match the non-motor vehicle with the same movement trajectory to the person to get the driver, or you can match the movement trajectory to the movement trajectory of the non-motor vehicle Personnel matching is driving personnel. For example, suppose that the motion information of non-motorized vehicle B is C, and that the motion information of two persons A1 and A2 are D1 and D2 respectively. When D1 and C are close to or the same, then person A1 can be regarded as a non-machine The driver of motor vehicle B, when A2 is closer to C than A1, then person A2 can be regarded as the driver of non-motor vehicle B.

It should be noted that the above-mentioned driver can also be called a driver or a driver.

In some possible embodiments, the non-motorized vehicle can also be matched with the occupants through the posture characteristics of the person. The posture characteristics can be posture characteristics such as hand posture, foot posture, upper body posture, etc. For example, by determining the Which person ’s posture features the hand belongs to to match the non-motor vehicle driver and passenger can be matched to the non-motor vehicle driver ’s passenger by judging which person ’s posture feature the foot on the pedal belongs to, etc. When the features are matched to the driver of the non-motorized vehicle, it is also necessary to match the posture feature to the person to determine the driver. The gesture feature can be associated with the face feature to match the gesture feature to the corresponding person.

104. According to the motion information of the non-motor vehicle, determine whether the non-motor vehicle triggers a preset illegal condition.

The motion information of the non-motorized vehicle may be speed information or trajectory information of the non-motorized vehicle during driving. The illegal conditions include: red light running conditions, motor vehicle lane occupying conditions, retrograde conditions, and speeding conditions.

The preset red light running condition can be a virtual coil, which is controlled by the road traffic signal control system, and the road traffic signal control system sends a snapshot signal to the camera. For example, the road traffic signal control system can be used when the traffic light is at the red light phase Send a snapshot signal to the camera, stop sending the snapshot signal to the camera when it is in the green light phase, or send the snapshot signal to the camera throughout the process. After the snapshot, set a virtual coil to determine whether the non-motor vehicle triggers the red light condition. The virtual coil is a common way of judging the red light, so I wo n’t go into details here.

In some possible embodiments, it can also be determined whether a non-motor vehicle triggers a red light running condition by burying a solid coil.

The preset conditions for occupying a motor vehicle lane may be to divide a motor vehicle lane and a non-motor vehicle lane, and to identify the existence state of a non-motor vehicle. .

The preset retrograde condition may be the value of the angle between the non-motor vehicle driving direction and the road direction. For example, if the angle between the non-motor vehicle driving direction and the road direction is more than 90 degrees, the non-motor vehicle may be judged to reverse and trigger the retrograde condition. In some possible implementation manners, a driving trajectory of a non-motor vehicle may be obtained, and when an angle between the driving trajectory and a road driving direction satisfies a retrograde judgment condition, a retrograde condition is triggered.

The preset overspeed condition can be the speed limit value of the non-motor vehicle lane of the current road section. The speed information of the non-motor vehicle can be obtained from the motion information of the non-motor vehicle. When the speed of the non-motor vehicle exceeds the speed limit value of the non-motor vehicle lane, Judge that the non-motor vehicle is speeding and trigger the speeding condition. For example, if the speed of the obtained non-motor vehicle is 40 yards, and the speed limit of the non-motor vehicle lane of the road section is 35 yards, it can be judged that the non-motor vehicle is speeding. In some possible implementations, microwave radar can be used. The speed sensor measures the speed of the non-motorized vehicle. When the speed of the non-motorized vehicle is measured, the speed of the non-motorized vehicle can be determined to trigger the overspeed condition.

It should be noted that the above-mentioned camera may be called a video camera or a camera, and the above-mentioned illegal conditions may also be called illegal conditions or illegal conditions.

105. If the non-motor vehicle triggers the preset illegal condition, confirm the identity information of the driver of the illegal non-motor vehicle.

In this step, the illegal conditions include: red light running conditions, occupied motorway conditions, retrograde conditions, speeding conditions, etc. In step 104, the above illegal conditions have been described accordingly, and will not be described here. In step 103, the driver and occupant of the non-motor vehicle has been matched, and the identity information can be confirmed through the facial features of the driver and occupant. For example, the face information of the driver and passenger is compared with a static database of ID cards. The static database of ID cards can be a remote database set in the public security system or a local database directly associated with the camera.

In some possible embodiments, after the identity information of the driver of the illegal non-motor vehicle is confirmed, the images of the illegal non-motor vehicle and its driver may also be retained as evidence of the violation.

It should be noted that the non-motor vehicle traffic violation supervision method provided by the embodiments of the present invention can be applied to non-motor vehicle traffic violation detection equipment, such as: traffic cameras, computers, servers, and other devices that can perform non-motor vehicle traffic violation detection.

In the embodiment of the present invention, image information is obtained, and the image information includes non-motor vehicle features of a non-motor vehicle and human face features; visually tracking the non-motor vehicle features and the face features to obtain the Non-motor vehicle movement information and the person's movement information; according to the non-motor vehicle movement information and the person's movement information for the non-locomotive matching driver and passenger; based on the non-motor vehicle movement information, It is determined whether the non-motor vehicle triggers a pre-set illegal condition; if the non-motor vehicle has illegal behavior, the identity information of the driver of the illegal non-motor vehicle is confirmed. Due to the matching of facial features and non-motor vehicle features, the non-motor vehicle drivers and passengers can be matched, and the traffic violations of non-motor vehicles can be more effectively supervised.

Please refer to FIG. 2. FIG. 2 is a schematic flowchart of another non-motor vehicle traffic violation supervision method provided by an embodiment of the present invention. As shown in FIG. 2, the method includes the following steps:

201. Acquire image information, where the image information includes non-motor vehicle features of a non-motor vehicle and facial features of a person.

202. Visually track the characteristics of the non-motor vehicle and the facial features to obtain the motion information of the non-motor vehicle and the motion information of the person.

203. Match the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel.

204. According to the motion information of the non-motor vehicle, determine whether the non-motor vehicle triggers a preset illegal condition.

205. If the non-motor vehicle triggers the preset illegal condition, confirm the identity information of the driver of the illegal non-motor vehicle.

206. Generate illegal information according to the illegal conditions triggered by the non-motor vehicle.

The aforementioned illegal information may be illegal types, wherein the illegal types may be red light running, occupation of vehicle lanes, retrograde, speeding and other illegal types. The aforementioned illegal types may be confirmed according to illegal conditions triggered by non-motor vehicles, for example, non-machine When a motor vehicle triggers a condition for running a red light, it generates illegal information for running a red light, and when a non-motor vehicle triggers a condition for occupying a motor vehicle lane, it generates illegal information for occupying a motor vehicle lane. Of course, the foregoing illegal information may include information such as illegal time and illegal location. For example, at time A and time B, a non-motor vehicle triggers a red light condition, then the illegal information of the non-motor vehicle running a red light at time A and B is generated. The above illegal information may also include the identity information of the driver and occupant. For example, if the driver C drives a red light condition when driving a non-motor vehicle at the time B at time A, then the driver C at the time B will drive the non-motor vehicle to drive a red light Illegal information. It should be noted that the above illegal information can be recorded or displayed through the captured face image. For example, by adding data such as illegal type, time, location and identity information to the face image to record or display, it can also be passed Use text to record or display, for example, to record or display the illegal type, time, location and identity information of illegal drivers and passengers through text. Of course, there may be other ways to record or display the illegal information, such as adding a illegal information label to a face image, or adding time information, location information, and identity information to the illegal video.

207. Generate prompt information according to the illegal information, and send the prompt information to a prompt device, where the prompt device is used to remind the driver of the illegal non-motor vehicle.

The above prompt information may be a prompt information generated according to the illegal type. For example, when driver C is driving at a time B at a time B and driving a non-motor vehicle to run a red light, a warning light may be generated: C, you drive at a time B at time A Non-motor vehicles running through red lights, please pay attention to pedestrians and driving safety. Or when the occupant C drives the non-motorized vehicle to occupy the motorized lane on the road B section at time A, a reminder message for the occupied motorized lane can be generated: C, you drive the non-motorized vehicle at the location A to occupy the motorized lane at time A, please pay attention to driving Safe, back to the non-motorized lane.

The above-mentioned prompting device may be a mobile device of a driver of a non-motor vehicle, wherein the mobile device may be a mobile device such as a mobile phone or a tablet provided with a prompt information receiving application, an electronic display sign installed at an intersection, or a setting Voice announcer on the roadside. You can send the prompt information to the driver's mobile device to prompt the driver or passenger, or you can send the prompt information to the electronic display board installed at the intersection to display the prompt information to remind the driver and passenger, you can also send the prompt information To the voice announcer installed on the roadside, the voice announcer of the voice announcer is used to make the driver and passengers get a reminder.

In the above steps, by generating illegal information, you can use the illegal information to record the driver of the illegal non-motorized vehicle, and by generating reminder information, you can warn the illegal non-motorized vehicle.

It should be noted that the non-motor vehicle traffic violation supervision method provided in the above embodiments can be applied to non-motor vehicle traffic violation detection equipment, such as: traffic cameras, computers, servers, and other devices that can perform non-motor vehicle traffic violation detection.

Please refer to FIG. 3, which is a schematic flowchart of another non-motor vehicle traffic violation supervision method provided by an embodiment of the present invention. As shown in FIG. 3, the method includes the following steps:

301. Acquire image information, the image information including non-motor vehicle features of a non-motor vehicle and human face features.

302. Perform visual tracking on the non-motor vehicle feature and the face feature to obtain the motion information of the non-motor vehicle and the person's motion information.

303. The motion information of the non-motor vehicle includes the motion information of the characteristics of the non-motor vehicle, and the motion information of the person includes the motion information of the facial characteristics;

Acquiring facial features within the preset range of the non-motor vehicle features, and matching the non-locomotive driver with the non-locomotive based on the motion information of the non-motor vehicle features and the facial feature motion information within the preset range .

304. According to the motion information of the non-motor vehicle, determine whether the non-motor vehicle triggers a preset illegal condition.

305. If the non-motor vehicle triggers the preset illegal condition, confirm the identity information of the driver of the illegal non-motor vehicle.

The motion information of the non-motor vehicle feature may be the speed or trajectory of the non-motor vehicle feature, and the motion information of the face feature may be the speed or trajectory of the face feature. In image recognition technology, the features to be recognized will be made into a feature box, for example, non-motor vehicle features and face features will be made into non-motor vehicle feature boxes and face feature boxes, respectively, the movement of non-motor vehicle features The information can be embodied as the motion information of the non-motor vehicle feature frame, and the facial feature motion information can be embodied as the facial feature frame motion information, which can be obtained by acquiring the facial feature frame within the preset range of the non-motor vehicle feature frame. Face features within the preset range of non-motor vehicle features. This preset range can be the pixel radius of the non-motor vehicle feature frame on the image, or the size radius. For example, you can obtain the face within 100 pixel radius of the non-motor vehicle feature Features, for example, can obtain a face feature frame whose distance from the center of the face feature frame to the center of the non-motor vehicle feature frame is less than 200 pixels. You can use the motion information of the non-motor vehicle features and the facial feature motion information within the preset range of the non-motor vehicle features to extract the facial features whose motion information is closest to or the same as the non-motor vehicle feature motion information. The facial features possessed by the driver and passenger can match the driver and passenger. For example, assuming that the non-motor vehicle feature is B, and the motion information of B is C, there are face features A1, A2, and A3 within the preset range of B, and the motion information of A1, A2, and A3 are D1, D2, and D3, respectively. Choose the one closest to C among D1, D2, and D3. If D1 is closest to C, the facial feature corresponding to D1 is A1, and the person matching facial feature A1 is the driver and occupant. D1, D2, D3 can be speed, trajectory, or distance.

It should be noted that step 203 is optional. For example, in some scenarios, it is only necessary to match the driver and passenger of the non-locomotive based on the motion information of the non-motor vehicle and the motion information of the personnel.

Match the driver and passenger to the non-locomotive according to the face set.

The motion information of the non-motor vehicle features may be the speed or trajectory of the non-motor vehicle features, and the motion information of the above face features may be the speed or trajectory of the face features. Face feature sets within the scope. Face sets can be identified with specific numbers or letter labels, such as face set E, face set 1, etc., or can be identified by feature values of non-motor vehicle features, such as non- If the feature of the motor vehicle is B, the face set is identified as face set B. It can also be identified by the color of the feature frame on the image. For example, if the feature frame of a non-motor vehicle feature on the image is green, it is preset. The feature boxes of the face features in the range are also green boxes, and the face set can be identified by the green representative mark G. In step 202, the non-motor vehicle features and the facial features are visually tracked, and multiple frames of images can be acquired. Through multi-frame images, it is possible to calculate the motion information of the non-motor vehicle features and the facial feature motion information, and within the preset range of the non-motor vehicle features, extract the face whose motion information is similar to the motion information of the non-motor vehicle features Features to form a face set. The preset range may be the pixel radius or the size radius on the image. The motion information of the face feature is similar to the motion information of the non-motor vehicle feature. It may be the threshold of the motion information of the non-motor vehicle feature, such as the speed of the non-motor vehicle feature. Threshold interval, when the speed of the face feature falls within the speed threshold interval of the non-motor vehicle feature, the motion information of the face feature and the non-motor vehicle feature can be regarded as similar. For example, if the speed threshold of non-motor vehicle features is 30 yards to 35 yards, when the speed of face features is 33 yards, the speed of face features and the speed of non-motor vehicle features can be regarded as similar, so that people The motion information of facial features is similar to that of non-motorized vehicles. The non-motor vehicle characteristic speed threshold interval can be set by the non-motor vehicle characteristic speed. The motion information of the facial features is similar to the motion information of the non-motor vehicle features, and may also be the threshold of the trajectory similarity of the non-motor vehicle features and the facial features. The degree threshold can be regarded as similar to the motion information of the face feature and the non-motor vehicle feature. For example: the threshold of the trajectory similarity of non-motor vehicle features and face features can be set to 80%. 'S motion information is similar to that of non-motorized vehicles.

In some possible embodiments, in each frame image, a face set can be formed, and multiple frame images can obtain multiple face sets, and the intersection of multiple face sets is obtained to obtain the final face set. This embodiment does not It is not limited that each frame of images forms a face set, or a partial frame of images may form a face set, such as a 200-frame image, and two frames may be extracted to form 2 face sets. The facial features in the final face set obtained by finding the intersection are the facial features that have always existed within the preset range of the non-motor vehicle features during visual tracking, and can also be said to be in the non-motor vehicle driving process and the non-motor vehicle process Keep facial features close to a certain distance. For example: suppose that 400 frames of images are acquired in visual tracking, and the 100 frames, 200 frames, 300 frames, and 400 frames of a total of 4 frames are extracted to form 4 human face sets E1, E2, E3, E4, and E1. Including facial features A1, A2, A3, A4, E2 includes facial features A1, A2, A4, A5, E3 includes facial features A1, A2, A5, A6, E4 includes facial features A1, A2, A4, A6 , Then the final face set E for the intersection of E1, E2, E3, E4 includes face features A1, A2, and the driver and passenger can be matched in the final face set E. At this time, you can use the face feature A1 Compare the motion information of A2 with the motion information of the non-motor vehicle features, and match the facial features of the motion information and the motion information of the non-motor vehicle features to the facial features possessed by the driver and passenger.

According to the speed comparison result, the face set is formed.

Among them, in step 202, the non-motor vehicle features and the facial features are visually tracked, and multiple frames of images can be acquired. Through multi-frame images, the speed of non-motor vehicle features and the speed of face features can be calculated. Within the preset range of non-motor vehicle features, face features with speeds similar to those of non-motor vehicle features are extracted to form a person Face set. The speed of non-motor vehicle features and the speed of face features can be calculated by the position change and time of the center point of the middle feature box in the image recognition technology. The preset range can be the pixel radius on the image or the radius of the size. The speed of the face feature is similar to the speed of the non-motor vehicle feature. It can be the speed threshold of the non-motor vehicle feature. For example, it can be the speed threshold interval of the non-motor vehicle feature. Compare the speed of the face feature with the speed threshold interval of the non-motor vehicle feature , When the speed of the face feature falls within the speed threshold interval of the non-motor vehicle feature, the speed of the face feature and the speed of the non-motor vehicle feature can be regarded as similar, for example: the speed threshold interval of the non-motor vehicle feature is 30 yards To 35 yards, assuming that the speed of face feature A1 is 33 yards, the speed of face feature A2 is 34 yards, and the speed of face feature A3 is 29 yards, you can compare the speeds of face features A1 and A2 with non-motor vehicle features The speed is considered to be similar, so that the facial features A1 and A2 are recorded into the face set, so that some people who are not clearly the driver of the motor vehicle can be removed For example, a person with a face feature A3 has only face features A1 and A2 in the face set, so that the face feature elements in the face set are reduced. When matching, only the face features A1 and A1 in the face set need to be calculated. A2, reduce the computational cost of matching driver and passengers through face sets. When the formed face set is an empty set, the face can be integrated into a non-empty face set by increasing the preset range of non-motor vehicle features and / or expanding the speed threshold interval of the non-motor vehicle features.

Optionally, the motion information of the non-motor vehicle feature includes a trajectory of the non-motor vehicle feature, and the motion information of the face feature includes a trajectory of the face feature;

Comparing the trajectory of the non-motor vehicle feature with the trajectory of the face feature to obtain a trajectory comparison result;

According to the trajectory comparison result, the face set is formed.

Among them, the trajectory comparison result can be the degree of coincidence between the trajectory of the face feature and the trajectory of the non-motor vehicle feature. For example, the center of the feature box of the face feature in the continuous image and the center of the feature box of the non-motor vehicle feature are The trajectories in the continuous image are compared starting from the same end point, and the ratio of the length of the trajectory of the facial feature and the trajectory of the non-motor vehicle feature to the total length of the trajectory of the facial feature and the trajectory of the non-motor vehicle feature is calculated to obtain the coincidence degree For example, suppose that the trajectory length of the face feature is 49, the trajectory length of the non-motor vehicle feature is 51, the length of the overlapped part is 45, and the coincidence degree is 45 * 2 / (49 + 51) * 100% = 90%. If the degree of coincidence is greater than 90% of the threshold of the degree of coincidence between the trajectory of the face feature and the trajectory of the non-motor vehicle feature, the face feature can be recorded into the face set.

In addition, the trajectory comparison result can also be the comparison result of the trajectory equation of the trajectory of the face feature and the trajectory of the non-motor vehicle feature. For example, the ratio or difference of the constants in the two trajectory equations, the closer the ratio is to 1, it means that The more similar the two trajectory equations, the closer the difference is to 0, indicating that the two trajectory equations are more similar.

Among them, the most recent pixel is the smallest pixel interval between the center of the feature frame of the face feature and the center of the feature box of the non-motor vehicle feature in the image, or the center of the feature box border of the face feature and the center of the feature box of the non-motor vehicle feature The pixel spacing between is the smallest.

In addition, if there are no multiple face features in the face set, that is to say, the face features in the face set are unique, it can be directly considered that the unique face features in the face set belong to the driver and occupant, thereby determining the driving Passengers.

In some possible embodiments, if there are multiple facial features in the face set, the facial features closest to the feature size of the non-motor vehicle can be selected from the multiple facial features to determine the facial features of the driver and passenger, and the non-machine Passengers in motor vehicles.

The facial features of the driver of the illegal non-motor vehicle are compared with the background identity image library to determine the identity information of the driver. As shown in Figure 4.

Image quality assessment (IQA) in the above is one of the basic technologies in image processing. It mainly analyzes and studies the characteristics of the image, and then evaluates the quality of the image (the degree of image distortion). The image quality score can be obtained through image quality evaluation. It can be set that the higher the image quality, the higher the image quality score, so that the image with the highest image quality score is selected. Compare the facial features extracted from the image with the highest image quality score with the ID photo in the background identity image library, select the ID photo with the highest similarity to the facial features, and extract the identity of the corresponding ID photo Information, so as to obtain the information of the drivers of illegal non-motor vehicles.

Of course, in some possible implementations, the facial features of all the people in the image with the highest quality score can also be extracted and compared with the ID card in the background identity image library to obtain the information of all the people in the image, and then Identify the driver.

The above background identity image library may be an image library set on a local server, or an image library set on a server on the cloud.

In this embodiment, a variety of optional implementations are added on the basis of the embodiment shown in FIG. 1, and the traffic violation confirmation rate of non-motor vehicles can be further improved.

Please refer to FIG. 5. FIG. 5 is a schematic structural diagram of a non-motor vehicle traffic violation supervision device provided by an embodiment of the present invention. As shown in FIG. 5, it includes:

The first acquisition module 401 is used to acquire image information, the image information includes non-motor vehicle features of the non-motor vehicle and human face features of the person;

The second obtaining module 402 is configured to perform visual tracking on the non-motor vehicle features and the face features, and obtain motion information of the non-motor vehicle and motion information of the person;

The matching module 403 is configured to match the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel;

The judging module 404 is used for judging whether the non-motor vehicle triggers a preset illegal condition according to the motion information of the non-motor vehicle;

The confirmation module 405 is configured to confirm the identity information of the driver of the illegal non-motor vehicle if the non-motor vehicle triggers the preset illegal condition.

Optionally, as shown in FIG. 6, the device further includes:

The first generating module 406 is configured to generate illegal information according to the illegal conditions triggered by the non-motor vehicle;

The second generation module 407 is configured to generate prompt information according to the illegal information, and send the prompt information to a prompt device, where the prompt device is used to remind the driver of the illegal non-motor vehicle.

Optionally, as shown in FIG. 5, the motion information of the non-motor vehicle includes motion information of the characteristics of the non-motor vehicle, and the motion information of the person includes motion information of the face characteristics;

The matching module 403 is used to obtain facial features within the preset range of the non-motor vehicle features, and is used to obtain motion information of the non-motor vehicle features and facial feature movement information within the preset range Match the driver to the non-locomotive.

Optionally, as shown in FIG. 7, the matching module 403 includes:

The processing unit 4031 is configured to form a face set according to the motion information of the non-motor vehicle characteristics and the motion information of the facial features within the preset range;

The matching unit 4032 matches drivers and occupants for the non-locomotive according to the face set.

Optionally, as shown in FIG. 8, the motion information of the non-motor vehicle feature includes the speed of the non-motor vehicle feature, and the motion information of the face feature includes the speed of the face feature;

The processing unit 4031 includes:

The first processing subunit 40311 compares the speed of the non-motor vehicle feature with the speed of the face feature to obtain a speed comparison result;

The second processing subunit 40312 forms the face set according to the speed comparison result.

Optionally, as shown in FIG. 9, the matching unit 4032 includes:

The detection subunit 40321 detects the number of face features in the face set, and determines whether there are multiple face features in the face set;

The determining sub-unit 40322, if there are multiple facial features, then select the facial features closest to the non-motor vehicle feature pixels among the multiple facial features to determine the facial features of the driver and passenger, to obtain the Drivers of non-motor vehicles.

Optionally, as shown in FIG. 10, the confirmation module 405 includes:

The image evaluation unit 4051 is configured to select images with facial features of the driver of the illegal non-motor vehicle from the acquired image information for quality evaluation, and obtain an image quality score;

The image selection unit 4052 is used to select the image with the highest image quality score according to the image quality score;

An extraction unit 4053 is used to extract the facial features of the driver of the illegal non-motor vehicle from the image with the highest image quality score;

The comparison unit 4054 is used to compare the facial features of the driver of the illegal non-motor vehicle with the background identity image library to determine the identity information of the driver.

It should be noted that the non-motor vehicle traffic violation supervision device provided by the embodiment of the present invention can be applied to non-motor vehicle traffic violation detection equipment, such as traffic cameras, computers, servers, and other devices that can perform non-motor vehicle traffic violation detection.

The non-motor vehicle traffic violation supervision device provided by the embodiment of the present invention can implement various implementation methods in the method embodiments of FIG. 1, FIG. 2, and FIG. 3, and corresponding beneficial effects. To avoid repetition, details are not described herein.

Referring to FIG. 11, FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. As shown in FIG. 11, it includes: a memory 902, a processor 901, and stored on the memory and can be on the processor A computer program that runs:

The processor 901 is used to call the computer program stored in the memory 902 and perform the following steps:

Optionally, after the processor 901 performs confirmation of the identity information of the driver of the illegal non-motor vehicle, the processor 901 further executes steps:

Prompt information is generated according to the illegal information, and the prompt information is sent to a prompting device, and the prompting device is used to remind the driver of the illegal non-motor vehicle.

The execution of the processor 901 according to the motion information of the non-motor vehicle and the motion information of the person to match the driver and passenger of the non-locomotive includes:

Optionally, the motion information according to the characteristics of the non-motor vehicle and the motion information of the face features within the preset range executed by the processor 901 for the non-locomotive matching driver and passenger include:

Match the driver and passenger to the non-locomotive according to the face set.

The execution of the processor 901 according to the motion information of the non-motor vehicle features and the motion information of the face features within the preset range to form a face set includes:

According to the speed comparison result, the face set is formed.

Optionally, the matching of the non-locomotive driver and passengers based on the face set by the processor 901 includes:

Optionally, the verification performed by the processor 901 to confirm the identity information of the driver of the illegal non-motor vehicle includes:

It should be noted that the electronic device provided by the embodiment of the present invention can be applied to non-motor vehicle traffic violation supervision equipment, such as: traffic cameras, computers, servers, and other devices that can detect non-motor vehicle traffic violation laws.

The electronic device provided by the embodiment of the present invention can implement various implementation methods in the method embodiments of FIG. 1, FIG. 2, and FIG. 3, and corresponding beneficial effects. To avoid repetition, details are not described herein again.

Embodiments of the present invention also provide a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to implement the non-motor vehicle traffic violation supervision method embodiment provided by the embodiment of the present invention. Each process can achieve the same technical effect. To avoid repetition, it will not be repeated here.

A person of ordinary skill in the art may understand that all or part of the processes in the method of the foregoing embodiments may be completed by instructing relevant hardware through a computer program, and the program may be stored in a computer-readable storage medium. During execution, the process of the above method embodiments may be included. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM for short), etc.

The above disclosure is only preferred embodiments of the present invention, and of course it cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope of the present invention.

Claims

A non-motor vehicle traffic illegal supervision method is characterized by including:

Obtain image information, the image information includes non-motor vehicle features of non-motor vehicles and human face features;

Visually tracking the characteristics of the non-motor vehicle and the facial features to obtain the movement information of the non-motor vehicle and the movement information of the person;

Matching driver and passengers for the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel;

According to the motion information of the non-motor vehicle, determine whether the non-motor vehicle triggers a preset illegal condition;

If the non-motor vehicle triggers the preset illegal condition, the identity information of the driver of the illegal non-motor vehicle is confirmed.
The method according to claim 1, wherein after the identification of the identity information of the driver of the illegal non-motor vehicle is performed, the method further comprises:

Generate illegal information according to the illegal conditions triggered by the non-motor vehicle;

Prompt information is generated according to the illegal information, and the prompt information is sent to a prompting device, and the prompting device is used to remind the driver of the illegal non-motor vehicle.
The method according to claim 2, wherein the motion information of the non-motor vehicle includes motion information of the characteristics of the non-motor vehicle, and the motion information of the person includes motion information of the face characteristics;

The matching the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel includes:

Acquiring facial features within the preset range of the non-motor vehicle features, and matching the non-locomotive driver with the non-locomotive based on the motion information of the non-motor vehicle features and the facial feature motion information within the preset range .
The method according to claim 3, wherein the motion information based on the non-motor vehicle characteristics and the facial feature motion information in the preset range for the non-locomotive matching driver and passengers includes :

Forming a face set according to the motion information of the non-motor vehicle features and the motion information of the face features within the preset range;

Match the driver and passenger to the non-locomotive according to the face set.
The method according to claim 3, wherein the motion information of the non-motor vehicle feature includes a speed of the non-motor vehicle feature, and the motion information of the face feature includes a speed of the face feature;

The forming of a face set according to the movement information of the non-motor vehicle characteristics and the movement information of the face characteristics within the preset range includes:

Comparing the speed of the non-motor vehicle feature with the speed of the face feature to obtain a speed comparison result;

According to the speed comparison result, the face set is formed.
The method according to claim 4, wherein matching the driver and the passenger for the non-locomotive based on the face set includes:

Detecting the number of face features in the face set to determine whether there are multiple face features in the face set;

If there are multiple facial features, the facial features closest to the non-motor vehicle feature pixels are selected from the multiple facial features to determine the facial features of the driver and occupant to obtain the driving characteristics of the non-motor vehicle Passengers.
The method according to any one of claims 1 to 6, wherein the confirmation of the identity information of the driver of the illegal non-motor vehicle includes:

From the acquired image information, select an image with the facial features of the driver of the illegal non-motor vehicle for quality evaluation, and obtain an image quality score;

According to the image quality score, select the image with the highest image quality score;

Extract the facial features of the driver of the illegal non-motor vehicle from the image with the highest image quality score;

The facial features of the driver of the illegal non-motor vehicle are compared with the background identity image library to determine the identity information of the driver.
A non-motor vehicle traffic illegal supervision device, characterized by including:

The first acquisition module is used to acquire image information, the image information includes non-motor vehicle features of the non-motor vehicle and human face features of the person;

A second acquisition module, configured to visually track the characteristics of the non-motor vehicle and the facial features, and obtain the motion information of the non-motor vehicle and the motion information of the person;

A matching module, configured to match the driver and passenger of the non-locomotive according to the motion information of the non-motor vehicle and the motion information of the personnel;

The judging module is used for judging whether the non-motor vehicle triggers a preset illegal condition according to the motion information of the non-motor vehicle;

The confirmation module is used for confirming the identity information of the driver of the illegal non-motor vehicle if the non-motor vehicle triggers the preset illegal condition.
An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the computer program as claimed in claim 1 To the steps in the non-motor vehicle traffic violation supervision method described in any one of 7.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the non-machine storage according to any one of claims 1 to 7 is realized Steps in the method of supervision of illegal traffic on motor vehicles.