WO2022063002A1 - Human-vehicle information association method and apparatus, and device and storage medium - Google Patents

Abstract

The embodiments of the present application relate to the field of artificial intelligence image identification. Disclosed are a human-vehicle information association method and apparatus, and a device and a storage medium The human-vehicle information association method of the present application comprises: acquiring a video to be processed; identifying a vehicle and a person comprised in said video, so as to obtain a vehicle identification box and a person identification box; respectively performing feature extraction on the vehicle identification box and the person identification box to obtain vehicle information and person information; determining, according to the vehicle identification box and the person identification box, whether there is an association between the vehicle and the person; and if so, associating the vehicle information with the person information.

Description

Person-vehicle information association method, device, device and storage medium

cross reference

This application is based on the Chinese patent application with the application number "202011009511.1" and the application date is September 23, 2020, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference. Apply.

technical field

The embodiments of the present application relate to the field of artificial intelligence image recognition, and in particular, to a method, device, device, and storage medium for associating person and vehicle information.

Background technique

With the development of artificial intelligence technology, intelligent video security systems have developed rapidly. At present, the intelligent video security system has become an important part of national informatization construction and national (community) security construction. At the same time, with the development of machine learning technology, vehicle, license plate, pedestrian, face and other recognition technologies based on deep learning are gradually mature. These technologies provide very convenient support for smart security work, and have also achieved great success in smart security work, and achieved outstanding success.

However, in the smart security system, the vehicle information about the vehicle and the person information about the user object are two completely independent parts, that is, there is no correlation between the vehicle and the person. Therefore, in order to identify related vehicles and people from massive video data, relevant staff need to spend a lot of time and energy.

Therefore, it is urgent to solve the problem of how to associate vehicles and personnel to reduce the time-consuming search and waste of human and material resources.

SUMMARY OF THE INVENTION

An embodiment of the present application provides a method for associating person and vehicle information, including: acquiring a video to be processed; identifying vehicles and persons included in the video to be processed, and obtaining a vehicle identification frame and a person identification frame; respectively identifying the vehicle identification frame and the person The frame performs feature extraction to obtain vehicle information and personnel information; according to the vehicle identification frame and the personnel identification frame, it is judged whether there is a relationship between the vehicle and the personnel; if there is an association, the vehicle information and personnel information are associated.

The embodiment of the present application also provides a person-vehicle information association device, including: an acquisition module for acquiring a video to be processed; an identification module for identifying vehicles and persons included in the to-be-processed video, and obtaining a vehicle identification frame and person identification The extraction module is used to perform feature extraction on the vehicle identification frame and the personnel identification frame respectively to obtain vehicle information and personnel information; the judgment module is used to judge whether the vehicle and the personnel are related according to the vehicle identification frame and the personnel identification frame; The module is used to associate the vehicle information with the personnel information when there is an association between the vehicle and the person.

Embodiments of the present application further provide a device for associating person and vehicle information, including: a memory communicatively connected to at least one processor; wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, So that at least one processor can execute the above method for associating person and vehicle information.

Embodiments of the present application further provide a computer-readable storage medium storing a computer program. When the computer program is executed by the processor, the above-mentioned method for associating person-vehicle information is realized.

Description of drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments.

1 is a flowchart of a method for associating person and vehicle information provided according to a first embodiment of the present application;

2 is a schematic diagram 1 of determining the positional relationship between a vehicle identification frame and a person identification frame in a method for associating person and vehicle information provided according to a second embodiment of the present application;

3 is a schematic diagram 2 of determining the positional relationship between a vehicle identification frame and a person identification frame in a method for associating person and vehicle information provided according to a second embodiment of the present application;

4 is a schematic diagram 3 of determining the positional relationship between a vehicle identification frame and a person identification frame in a method for associating person-vehicle information provided according to the second embodiment of the present application;

5 is a schematic diagram 4 of determining the positional relationship between a vehicle identification frame and a person identification frame in a method for associating person and vehicle information provided according to the second embodiment of the present application;

6 is a schematic diagram of determining whether a vehicle and a person are associated in a method for associating person-vehicle information provided according to a third embodiment of the present application;

7 is a schematic diagram of determining whether a vehicle and a person are associated in a method for associating person-vehicle information provided according to a fourth embodiment of the present application;

8 is a schematic structural diagram of a device for associating person-vehicle information provided according to a fifth embodiment of the present application;

FIG. 9 is a schematic structural diagram of a device for associating person-vehicle information provided according to a sixth embodiment of the present application.

detailed description

In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that, in each embodiment of the present application, many technical details are provided for the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be realized. The following divisions of the various embodiments are for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the various embodiments may be combined with each other and referred to each other on the premise of not contradicting each other.

The purpose of the embodiments of the present application is to provide a method, device, device, and storage medium for associating person-vehicle information, so as to solve the above-mentioned technical problems.

The method, device, device and storage medium for the association of human and vehicle information proposed in this application can effectively avoid the interference of background factors by extracting features from the vehicle identification frame and the personnel identification frame calibrated in the video to be processed, thereby ensuring that the extracted The accuracy of the vehicle information and personnel information; judge whether the corresponding vehicle and personnel are related according to the calibrated vehicle identification frame and personnel identification frame, and when it is determined that the two are related, the isolated vehicle information and personnel information are analyzed. In the subsequent application scenarios of finding a person by car or finding a car by person, the time-consuming search and the waste of manpower and material resources are greatly reduced.

The first embodiment relates to a method for associating person and vehicle information. By performing feature extraction from the vehicle identification frame and the person identification frame demarcated in the video to be processed, the interference of background factors is effectively avoided, thereby ensuring that the extracted vehicle information and Accuracy of personnel information; by judging whether the corresponding vehicle and personnel are related according to the calibrated vehicle identification frame and personnel identification frame, and when it is determined that the two are related, the isolated vehicle information and personnel information are associated to achieve In the subsequent application scenarios of finding people by cars or finding cars by people, the time-consuming search and the waste of manpower and material resources are greatly reduced.

The implementation details of the method for associating person-vehicle information in this embodiment are described below. The following content is only provided for the convenience of understanding, and is not necessary for implementing this solution.

The method for associating person and vehicle information provided in this embodiment is specifically applied to any terminal device capable of executing the method, such as a personal computer, a tablet computer, a smart phone, etc., which are not listed one by one here, and this embodiment does not limit this. .

The specific process of this embodiment is shown in Figure 1, which specifically includes the following steps:

Step 101, acquiring the video to be processed.

Specifically, the above-mentioned to-be-processed video may be from surveillance cameras set up in different places, or may be from various big data platforms, which is not limited in this embodiment.

In addition, the above-mentioned to-be-processed videos may be videos of various formats and forms, which are also not limited in this embodiment.

Step 102 , identify vehicles and persons included in the video to be processed, and obtain a vehicle identification frame and a person identification frame.

It should be understood that, in the actual operation, the feature extraction of the target object usually first determines the target object, then calibrates the target object, and then determines the position of the target object in the video to be processed, so as to obtain the target object of the target object. Identify the frame, and then perform feature extraction on the target object in the target recognition frame by using a preset recognition technology.

Therefore, in order to realize the association of human-vehicle information, it is necessary to adopt a preset recognition technology. This embodiment is based on the artificial intelligence video recognition technology. Through the analysis and processing of the video to be processed, the vehicles and people included in the video to be processed are identified. Finally, the corresponding vehicle identification frame and person identification frame are calibrated according to the identified vehicles and people.

The above-mentioned vehicles are not limited to motor vehicles in this embodiment. Other vehicles with obvious characteristics, such as electric vehicles, tricycles, and rickshaws, are all vehicles that need to be identified from the video to be processed.

Correspondingly, the above-mentioned persons include not only drivers and passengers driving or riding the above-mentioned vehicles, but also pedestrians, that is, all persons appearing in the video to be processed belong to the vehicle that needs to be identified from the video to be processed.

For ease of understanding, this embodiment provides an implementation method for identifying vehicles and people from the video to be processed, and then marking the vehicle identification frame corresponding to the vehicle and the personnel identification frame corresponding to the person, as follows:

(1) Extract a video frame image from the video to be processed, that is, take the frame as a unit, take the image corresponding to each frame as a video frame image, or take the images corresponding to certain frames as a video frame image.

Specifically, in practical applications, when the video picture to be processed has a small change, that is, dozens of frames or even more frames correspond to the same picture, at this time, you can choose to combine these frames, that is, the frames corresponding to the same picture into one frame to obtain a video frame image.

Correspondingly, when the video picture to be processed changes greatly, that is, each frame or every few frames corresponds to a different picture, at this time, one frame can be selected to correspond to one video frame image.

(2) Vehicle detection is performed on the video frame image, and when a vehicle is detected, all vehicles appearing in the video frame image are calibrated in the video frame image, and N vehicle identification frames are obtained.

(3) Perform face and human shape detection on the video frame image, and when a person is detected, all persons appearing in the video frame image are calibrated in the video frame image, and M person identification frames are obtained.

It should be understood that since a video to be processed can usually extract multiple video frame images, in a specific application, it is necessary to perform the above-mentioned vehicle detection and face and human shape detection on each video frame image.

In order to facilitate implementation, a preset machine learning algorithm, such as a deep convolutional neural network algorithm, can be used to train vehicle sample data and face and humanoid sample data respectively, and then obtain a vehicle recognition model for recognizing vehicles and a vehicle recognition model for recognizing people. face and humanoid recognition model.

In addition, it should be understood that, due to the relative popularity of image recognition technology, the specific recognition method will not be repeated in this embodiment, and those skilled in the art can select an appropriate recognition technology to recognize the video to be processed according to needs, so as to identify the video to be processed. Vehicles and people included in the video, and then determine the corresponding vehicle identification frame and person identification frame.

In addition, both N and M mentioned above are integers greater than 0, and in practical applications, the values of N and M may be the same or different.

Step 103 , respectively perform feature extraction on the vehicle identification frame and the personnel identification frame to obtain vehicle information and personnel information.

Specifically, in this embodiment, the vehicle information extracted from the vehicle identification frame mainly includes license plate number, model, vehicle color, vehicle brand, vehicle accessories (such as body, interior decoration, etc.), vehicle travel direction, etc. , vehicle appearance time, etc.

Correspondingly, the personnel information extracted from the personnel identification frame mainly includes facial feature information, clothing color, clothing style, hairstyle, hair color, body accessories (such as watches, backpacks, eyes, etc.), personnel travel direction, personnel appearance, etc. time etc.

In addition, it can be known from the description of step 102 that there may be multiple vehicle identification frames and person identification frames identified from the video to be processed. Therefore, in order to determine the correlation between each vehicle and each person in the video to be processed, you can First, the identification frames corresponding to these vehicles and people are combined, that is, N vehicle identification frames and M personnel identification frames are formed, and then an N×M personal vehicle combination is obtained.

After the N×M personal-vehicle combinations are obtained, the N×M personal-vehicle combinations need to be traversed, that is, the feature extraction operation mentioned in step 103 needs to be performed for each traversed personal-vehicle combination.

Step 104 , according to the vehicle identification frame and the person identification frame, determine whether there is a relationship between the vehicle and the person.

Specifically, if it is determined that there is a relationship between the vehicle and the person through the judgment, then go to step 105, otherwise it ends directly, or a prompt is given, for example, the vehicle and the person in the currently judged person-vehicle combination are not related.

In this embodiment, when judging whether the vehicle and the person in the current person-vehicle combination are related according to the vehicle identification frame and the person identification frame extracted from each person-vehicle combination, the specific method is to obtain the coordinate information of the vehicle identification frame and the person. The coordinate information of the personnel identification frame is obtained, and then the vehicle coordinate information and the personnel coordinate information are obtained; finally, whether the vehicle and the personnel are related is judged according to the vehicle coordinate information and the personnel coordinate information.

The vehicle coordinate information mentioned above may be the coordinate information of the four vertices of the vehicle identification frame, specifically the coordinate value of the corresponding vertex, that is, the value corresponding to the X axis and the value corresponding to the Y axis, or the vertex in the upper left corner. The coordinate value and the vertex coordinate value of the lower right corner can also be the vertex coordinate value of the upper right corner and the vertex coordinate value of the lower left corner.

Correspondingly, the personnel coordinate information can be the coordinate information of the four vertices of any recognition frame, the vertex coordinate value of the upper left corner and the vertex coordinate value of the lower right corner, or the vertex coordinate value of the upper right corner and the vertex of the lower left corner. Coordinate value.

In addition, it is worth mentioning that, in order to facilitate the determination of whether the vehicle and the person are related according to the vehicle coordinate information and the personnel coordinate information, in a specific application, if the vehicle coordinate information and the personnel coordinate information both include only two vertex coordinate values, it is necessary to Ensure that the vertex coordinate values included in the vehicle coordinate information and the personnel coordinate information are the vertex coordinate values of the corresponding vertices. That is, if the vehicle coordinate information is the vertex coordinate value of the upper left corner and the vertex coordinate value of the lower right corner, the personnel coordinate information also needs to be the upper left corner. The vertex coordinate value of the corner and the vertex coordinate value of the lower right corner. Conversely, if the vehicle coordinate information is the vertex coordinate value of the upper right corner and the vertex coordinate value of the lower left corner, the personnel coordinate information also needs to be the vertex coordinate value of the upper right corner and the vertex coordinate value of the lower left corner.

For convenience of description, in this embodiment, the vehicle coordinate information at least includes the vertex coordinate value L ₁ of the upper left corner of the vehicle identification frame and the vertex coordinate value R ₁ of the lower right corner, and the personnel coordinate information at least includes the vertex coordinate value L ₂ of the upper left corner of the personnel identification frame. and the vertex coordinate value R ₂ of the lower right corner as an example to illustrate:

Specifically, _first , the vertex coordinate value L1 is compared with the vertex coordinate value L2, and the vertex coordinate value _R1 is compared with the vertex coordinate value R2 _; then, the positions _of the vehicle identification frame and the person identification frame are determined according to the comparison result. relationship; finally, according to the position relationship, it is judged whether there is a relationship between the vehicle and the person.

That is to say, whether the vehicle and the person are related is determined according to the positional relationship between the vehicle identification frame where the vehicle is located and the personnel identification frame where the person is located.

For example, when the positional relationship between the vehicle identification frame and the person identification frame is an inclusive relationship, it may be considered that the vehicle in the vehicle identification frame and the person in the person identification frame are related.

For another example, when the positional relationship between the vehicle identification frame and the person identification frame is overlapping, that is, not inclusive, but partially overlapping, it can be considered that the vehicle in the vehicle identification frame and the person in the person identification frame are related.

For another example, the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship, that is, the two are neither contained nor overlapped. It can be considered that the vehicle in the vehicle identification frame and the person in the personnel identification frame are not related. .

Step 105, associate the vehicle information with the personnel information.

Specifically, when there is an association between the vehicle in the vehicle identification frame and the person in the person identification frame, the operation of associating the vehicle information and the personnel information in step 105 is specifically to associate the license plate number and model of the vehicle information. , vehicle color, vehicle brand, vehicle accessories (such as body, interior decoration, etc.), vehicle travel direction, vehicle appearance time, etc. and face feature information in personnel information, clothing color, clothing style, hairstyle, hair color, Physical attachments (such as watches, backpacks, eyes, etc.), the direction of travel of the person, the time of the person's appearance, etc. are associated.

That is, in practical applications, as long as any one or any of the above vehicle information is extracted, all vehicle information and personnel information of related personnel can be found out, thereby greatly shortening the retrieval and screening time. , and then in security scenarios, such as people looking for cars or cars looking for people, the time-consuming search and the waste of manpower and material resources are greatly reduced.

For example, for a certain person Zhang San, there is a black Audi Q5 with the license plate number "123456". If Zhang San drives this car at location A at 2:00 pm on September 14, 2020, just enter the license plate number "123456", or Zhang San's ID number, or his biometric information, you can get the following person-car association information: Zhang San with ID number XXXX drives a black Audi Q5 with license plate number 123456 in September 2020 Appeared at location A at 2 pm on the 14th.

It should be understood that the above examples are only examples listed for better understanding of the technical solutions of the present embodiment, and are not used as the only limitation on the present embodiment.

In practical applications, vehicle information and personnel information can be reasonably extracted according to usage scenarios, and corresponding systems, such as smart security systems, can be built as needed to provide more convenient services for relevant staff.

It is not difficult to find from the above description that the method for associating person and vehicle information provided in this embodiment effectively avoids the interference of background factors by extracting features from the vehicle identification frame and the person identification frame demarcated in the video to be processed, thereby ensuring the extraction of features. The accuracy of the vehicle information and personnel information released.

In addition, by judging whether the corresponding vehicle and person are related according to the calibrated vehicle identification frame and person identification frame, and when it is determined that the two are related, the isolated vehicle information and personnel information are associated, so as to realize the human-vehicle information. In the subsequent application scenarios of finding people by cars or finding cars by people, the time-consuming search and the waste of manpower and material resources are greatly reduced.

The second embodiment of the present application relates to a method for associating person and vehicle information. The second embodiment is mainly aimed at a specific application scenario in which the positional relationship between the vehicle identification frame and the person identification frame is determined according to the comparison result mentioned in the first embodiment. illustrate.

Specifically, since the comparison result used to determine the positional relationship between the vehicle identification frame and the person identification frame is performed by comparing the vertex coordinate value _L1 of the vehicle identification frame with the vertex coordinate value L2 _of the person identification frame, The vertex coordinate value R ₁ of the frame is obtained by comparing the vertex coordinate value R ₂ of the person identification frame. Therefore, there are the following comparison results: ① The vertex coordinate value L ₁ is greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is greater than the vertex coordinate value R ₂ ; ② The vertex coordinate value L ₁ is greater than the vertex coordinate value L ₂ , and the vertex coordinate value The value R ₁ is equal to the vertex coordinate value R ₂ ; ③ the vertex coordinate value L ₁ is greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is smaller than the vertex coordinate value R ₂ ; ④ The vertex coordinate value L ₁ is equal to the vertex coordinate value L ₂ , and the vertex coordinate value Value R ₁ is greater than vertex coordinate value R ₂ ; ⑤ vertex coordinate value L ₁ is equal to vertex coordinate value L ₂ , vertex coordinate value R ₁ is equal to vertex coordinate value R ₂ ; ⑥ vertex coordinate value L ₁ is equal to vertex coordinate value L ₂ , vertex coordinate value Value R ₁ is less than vertex coordinate value R ₂ ; ⑦ vertex coordinate value L ₁ is less than vertex coordinate value L ₂ , vertex coordinate value R ₁ is greater than vertex coordinate value R ₂ ; ⑧ vertex coordinate value L ₁ is less than vertex coordinate value L ₂ , vertex coordinate value The value R ₁ is equal to the vertex coordinate value R ₂ ; ⑨ the vertex coordinate value L ₁ is smaller than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is smaller than the vertex coordinate value R ₂ .

It can be seen from the above nine comparison results that only when the cases are ④, ⑤, ⑦, ⑧, that is, the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not less than the vertex coordinate value R ₂ , when The positional relationship between the vehicle identification frame and the person identification frame is an inclusion relationship, specifically, the vehicle identification frame includes the personnel identification frame.

For ease of understanding, this embodiment takes case ⑦ as an example, and the positional relationship between the vehicle identification frame and the person identification frame is shown in FIG. 2 .

Correspondingly, when the situation is ②, ③, ⑤, ⑥, that is, the vertex coordinate value L ₁ is not less than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , the vehicle identification frame and the person are identified. The positional relationship of the boxes is an inclusion relationship, specifically, the person identification box includes the vehicle identification box.

For ease of understanding, this embodiment takes case ③ as an example, and the positional relationship between the vehicle identification frame and the person identification frame is shown in FIG. 3 .

It should be understood that, for the above situation ⑤, that is, the situation in which the vehicle identification frame and the person identification frame completely overlap, it is also an inclusion relationship in essence.

Correspondingly, in cases ① and ⑨, it can be determined whether the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship.

For the operation of determining the positional relationship between the vehicle identification frame and the person identification frame according to the nine situations given above, in practical applications, it can be implemented by the following pseudocode:

It should be understood that the above examples are only examples listed for better understanding of the technical solutions of the present embodiment, and are not used as the only limitation on the present embodiment.

In addition, it is worth mentioning that in general, the vehicle identification frame should be larger than the person identification frame, that is, the person is located in the car, which may be the driver or the passenger, that is, the situation given in Figure 2. However, in the actual processing process of the video to be processed, there may be people outside the vehicle, but the distance from the vehicle is too close, such as close to the vehicle, the person identification frame circled at this time often circles the vehicle. It will appear that the person identification frame is larger than the vehicle identification frame, that is, the situation given in Figure 3.

Further, after determining whether the positional relationship between the vehicle identification frame and the personnel identification frame is a non-inclusive relationship, that is, through comparison, the obtained comparison results are the situations ① and ⑨ given above, and the following two judgment logics can be used to further judge the occurrence of Whether there is an association between vehicles and people in the same image frame.

Method 1: Determine whether the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and whether the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , and whether the vertex coordinate value L ₂ is less than the vertex coordinate value R ₁ .

Correspondingly, if the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , and the vertex coordinate value L ₂ is less than the vertex coordinate value R ₁ , the vehicle identification frame and the vertex coordinate value are determined. The positional relationship of the personnel identification frame is an overlapping relationship, that is, the situation shown in Figure 4; otherwise, the vehicle identification frame and the personnel identification frame are completely separated, and there is no overlapping area between the two, and the positional relationship between the vehicle identification frame and the personnel identification frame is determined. Is a non-inclusive and non-overlapping relationship.

Method 2: Determine whether the vertex coordinate value L ₂ is not greater than the vertex coordinate value L ₁ , and whether the vertex coordinate value R ₂ is not greater than the vertex coordinate value R ₁ , and whether the vertex coordinate value L ₁ is less than the vertex coordinate value R ₂ .

Correspondingly, if the vertex coordinate value L ₂ is not greater than the vertex coordinate value L ₁ , and the vertex coordinate value R ₂ is not greater than the vertex coordinate value R ₁ , and the vertex coordinate value L ₁ is less than the vertex coordinate value R ₂ , the vehicle identification frame and the vertex coordinate value are determined. The positional relationship of the personnel identification frame is an overlapping relationship, that is, the situation shown in Figure 5; otherwise, the vehicle identification frame and the personnel identification frame are completely separated, and there is no overlapping area between the two, and the positional relationship between the vehicle identification frame and the personnel identification frame is determined. Is a non-inclusive and non-overlapping relationship.

In addition, it is worth mentioning that, in order to ensure the accuracy of the judgment results as much as possible, before determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship, it is also possible to judge whether the overlapping area satisfies the preset overlapping conditions.

Correspondingly, if it is satisfied, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship; otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.

Regarding the above-mentioned operation of judging whether the overlapping area satisfies the preset overlapping condition, the logic for Mode 1 and Mode 2 is roughly the same, except that the vertex coordinate values on which the overlapping area and the reference area are determined are different.

Specifically, for Mode 1, the specific implementation is as follows:

First, according to the vertex coordinate value L ₂ and the vertex coordinate value R ₁ , determine the overlapping area, as shown in the shaded part in Figure 4 ; according to the vertex coordinate value L ₁ and the vertex coordinate value R ₂ , determine the reference area, as shown in Figure 4 part of the dashed box.

Then, it is determined whether the ratio of the overlapping area to the reference area is greater than the first preset threshold.

Finally, if it is greater than that, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship; otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.

For mode 2, the specific implementation is as follows:

First, according to the vertex coordinate value R ₂ and the vertex coordinate value L ₁ , determine the overlapping area, as shown in the shaded part in Fig. 5 ; according to the vertex coordinate value L ₂ and the vertex coordinate value R ₁ , determine the reference area, as shown in Fig. 5 part of the dashed box.

Then, it is determined whether the ratio of the overlapping area to the reference area is greater than the first preset threshold.

Finally, if it is greater than that, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship; otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.

Regarding the above-mentioned first preset threshold, it can be set by those skilled in the art according to actual needs, for example, it is set to 20%.

For the operation of further judging whether the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship according to the overlapping area, in practical applications, it can be implemented by the following pseudocode:

It should be understood that the above examples are only examples listed for better understanding of the technical solutions of the present embodiment, and are not used as the only limitation on the present embodiment.

In addition, in practical applications, in order to ensure the accuracy of the judgment results as much as possible, for each of the above judgments, the video frame images corresponding to adjacent frames can be used to assist the judgment to eliminate interference and ensure the final establishment of the human-vehicle association. accuracy of results.

In addition, in practical applications, when the above calculation processing is performed according to the coordinate values of each vertex, the calculation is specifically performed according to the coordinate values of each vertex on the X axis and the Y axis.

Therefore, the positional relationship between the vehicle identification frame and the person identification frame is determined according to the above method, so that the person-vehicle information association method provided in this embodiment can be adapted to the same frame as the person and the vehicle (the person is on the vehicle), the person and the vehicle Scenes with different frames but the same painting can better analyze the relationship between isolated and scattered target information (vehicle information and personnel information). For example, although a user is not the owner of a vehicle, he often When the vehicle appears in the same place at the same time, when the vehicle needs to be tracked later, it is possible to track the vehicle as long as the travel trajectory of the user is obtained, thus providing more, deeper and more valuable information for the relevant staff. Clue information, thereby improving the practical value of vehicle information and personnel information.

The third embodiment of the present application relates to a method for associating person and vehicle information. The third embodiment is mainly aimed at determining the non-inclusive and non-overlapping relationship between the vehicle identification frame and the person identification frame as mentioned in the second embodiment, that is, there is a certain distance between the vehicle identification frame and the personnel identification frame. There is no scene before association, that is, the third embodiment is to further filter out related vehicles and personnel from non-associated vehicles and personnel. For ease of understanding and description, the following description is made with reference to FIG. 6 .

As shown in FIG. 6 , before it is determined that there is no relationship between the vehicle and the person, the coordinate value C ₁ of the center point of the vehicle identification frame can be determined according to the vertex coordinate value L ₁ and the vertex coordinate value R ₁ ; according to the vertex coordinate value L ₂ and The vertex coordinate value R ₂ determines the center point coordinate value C ₂ of the personnel identification frame; then, according to the center point coordinate value C ₁ and the center point coordinate value C ₂ , the distance from the center point of the vehicle identification frame to the center point of the personnel identification frame is determined. distance D ₁ ; then, determine the diagonal distance D ₂ of the video frame image; then, determine whether the ratio of the distance D ₁ to the diagonal distance D ₂ is greater than the second preset threshold; The person is associated; otherwise, it is determined that the vehicle and the person are not associated.

Regarding, the above-mentioned diagonal distance D ₂ may be determined according to the vertex coordinate value of the upper left corner of the video frame image and the vertex coordinate value of the lower right corner in practical application, or may be determined according to the upper right corner of the video frame image. The vertex coordinate value and the vertex coordinate value of the lower left corner are determined, which is not limited in this embodiment.

In addition, in practical applications, the distance D ₁ can be determined according to the center point coordinate value C ₁ of the vehicle identification frame and the center point coordinate value C ₂ of the personnel identification frame, and can also be determined according to the vehicle identification frame or the vehicle identification frame. , which is determined with the coordinate value of the coordinate point on the personnel identification frame or in the personnel identification frame, which is not limited in this embodiment.

In addition, the above-mentioned second preset threshold can be set by those skilled in the art according to actual needs, for example, it is set to 30%.

For the operation of further judging whether there is _a relationship between the person in the vehicle identification frame and the person in the person identification frame according to the distance D1 and the diagonal distance _D2 , in practical applications, the following pseudocode can be used to implement:

It should be understood that the above examples are only examples listed for better understanding of the technical solutions of the present embodiment, and are not used as the only limitation on the present embodiment.

Similarly, in practical applications, in order to ensure the accuracy of the judgment results as much as possible, for each of the above judgments, the video frame images corresponding to adjacent frames can be used to assist the judgment to eliminate interference, thereby ensuring the final establishment of the human-vehicle association. accuracy of results.

Therefore, before it is determined that the vehicle identification frame and the person identification frame are non-inclusive and non-overlapping, that is, there is a certain distance between the vehicle identification frame and the personnel identification frame, and before it is determined that the vehicle and the person are not associated, the vehicle identification is judged by the distance. Whether there is a relationship between the vehicle in the frame and the person in the person identification frame, so that the method for associating the person and vehicle information provided in this embodiment can be better adapted to the scene where the person and vehicle are in different frames but the same picture, and thus can better isolate the , scattered target information (vehicle information and personnel information) to analyze the relationship between them.

The fourth embodiment of the present application relates to a method for associating person and vehicle information. _The fourth embodiment is mainly aimed at the scene in the third embodiment when it is determined that the vehicle and the person are not related according to the distance D1 and the diagonal distance D2, that is, the person and the vehicle are _on the same screen, but the distance judgment is not satisfied. , that is, the fourth embodiment is to further filter out related vehicles and persons from unrelated vehicles and persons. For ease of understanding and description, the following description is made with reference to FIG. 7 .

Specifically, before determining that there is no association between the vehicle and the person according to the distance D ₁ and the diagonal distance D ₂ , the following operations may be performed first:

(1) Expand the vehicle identification frame and the person identification frame to obtain the background identification frame.

Specifically, the above-mentioned operation of expanding the vehicle identification frame and the person identification frame is to expand the vehicle identification frame and the person identification frame outward to a preset size.

In addition, in practical applications, the above-mentioned background identification frame may include a vehicle identification frame and a person identification frame; it may also be expanded for the vehicle identification frame and the personnel identification frame respectively, so as to obtain the background corresponding to the vehicle identification frame. The background identification frame corresponding to the identification frame and the person identification frame.

In order to facilitate processing, this embodiment adopts the first method, that is, the background recognition frame includes a vehicle recognition frame and a person recognition frame.

(2) Feature extraction is performed on the background recognition frame, and the location information of vehicles and persons is determined according to the extracted features.

Regarding the feature extraction operation for the background recognition frame, those skilled in the art can pre-select a suitable machine learning algorithm to construct a corresponding feature extraction model as needed, and then perform feature extraction on the background recognition frame according to the feature extraction model obtained by training, so as to extract The characteristic information that meets the requirements.

(3) Determine the time information of vehicles and people appearing in the video to be processed.

(4) Acquire an associated video associated with the video to be processed.

Specifically, the associated video is a video captured by a camera located in the same area as the camera that shoots the video to be processed, but located at a different point.

(5) According to the location information and time information, extract the associated video frame image from the associated video provided by each point.

The operation of extracting the associated video frame image from the associated video provided by each point in step (5), and then identifying the above-mentioned vehicles and personnel in the associated video frame image, is the same as that of step 102 in the first embodiment. The extraction method given in is similar, and will not be repeated here.

(6) Traverse each associated video frame image, and record the point where the vehicle appears and the point where the person appears.

Correspondingly, if the intersection of the point where the vehicle appears and the point where the person appears is greater than the third threshold, it is determined that the vehicle and the person are associated; otherwise, it is determined that the vehicle and the person are not associated.

The above-mentioned third preset threshold can be set by those skilled in the art according to actual needs, for example, it is set to be greater than half of the number of points.

For ease of understanding, the following description is given in conjunction with Figure 7:

Assuming that the correlation between vehicle A and person B needs to be judged, after the judgments given in the first, second and third embodiments above, it is still impossible to determine the correlation between vehicle A and person B, then use the When the method given in this embodiment is further determined, the associated video 1 captured by the camera at point 1, the associated video 2 captured by the camera at point 2, the associated video 3 captured by the camera at point 3, and the associated video at Associated video 4 taken by the camera at point 4.

According to the location information determined in the above (2) and the time information determined in step (3), the corresponding video frame images are extracted from the associated video 1, associated video 2, associated video 3 and associated video 4, respectively, and then record the point where the vehicle appears. position and the point of presence of personnel.

As shown in Figure 7, vehicle A and person B appear in the video frame image captured at point 1, vehicle A and person B appear in the video frame image captured at point 2, and only in the video frame image captured at point 3 Person B appears, and vehicle A and person B appear in the video frame image captured at point 4.

Based on the above regulations, the intersection of the point where the vehicle A appears and the point where the person B appears is greater than the third threshold, if it is greater than half of the number of points, it can be seen from the above records that there are 4 auxiliary points in Figure 7 when there are 4 , as long as the intersection of the point where vehicle A appears and the point where person B appears is greater than 2, that is, 3 or 4, it can be considered that vehicle A and person B are related, that is, in the example given in Figure 7, vehicle A and person B exist association.

In addition, it is worth mentioning that due to the high probability of a single misjudgment, in order to ensure the accuracy of the result determined based on the above method, the number of such simultaneous occurrences is required to be multiple and frequent.

Therefore, when the person and the vehicle are on the same screen, but the distance judgment is not satisfied, the to-be-processed video shot by the cameras at different points is used to assist in determining whether the current vehicle and the person are related, so that the person provided by this embodiment is provided. The vehicle information association method can be better adapted to the scene where people and vehicles are in the same picture, but does not meet the distance judgment, that is, the related vehicles and people can be screened out from the scene as much as possible, so as to better isolate and scattered. The target information (vehicle information and personnel information) is analyzed to analyze the relationship between them.

In addition, it should be understood that the division of steps of the various methods above is only for the purpose of describing clearly, and can be combined into one step or split into some steps during implementation, and decomposed into multiple steps, as long as the same logical relationship is included, all Within the protection scope of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of the algorithm and process are all within the protection scope of this patent.

The fifth embodiment of the present application relates to a device for associating person and vehicle information, as shown in FIG.

Among them, the acquisition module 801 is used to acquire the video to be processed; the identification module 802 is used to identify the vehicles and people included in the video to be processed, and obtain the vehicle identification frame and the personnel identification frame; the extraction module 803 is used to respectively identify the vehicle identification frame Perform feature extraction with the personnel identification frame to obtain vehicle information and personnel information; the judgment module 804 is used to judge whether the vehicle and personnel are related according to the vehicle identification frame and the personnel identification frame; the association module 805 is used for the existence of association between vehicles and personnel. Yes, associate vehicle information with personnel information.

In addition, in another example, when the identification module 803 identifies the vehicles and people included in the video to be processed, and obtains the vehicle identification frame and the personnel identification frame, the specific steps are:

Extract video frame images from the video to be processed;

Vehicle detection is performed on the video frame image, and when a vehicle is detected, all vehicles appearing in the video frame image are calibrated in the video frame image, and N vehicle identification frames are obtained, where N is an integer greater than 0;

Perform face and human shape detection on the video frame image, and when a person is detected, all persons appearing in the video frame image are calibrated in the video frame image, and M person identification frames are obtained, where M is an integer greater than 0.

In addition, in another example, the device for associating person-vehicle information may further include a combination module.

Specifically, the combination module is used to combine the N vehicle identification frames and the M personnel identification frames to obtain an N×M personal vehicle combination.

Accordingly, the operations performed by the determination module 804 are adapted to each person-vehicle combination. That is, the judging module 804 performs the above-mentioned judging operation for each traversed personal-vehicle combination by traversing N×M personal-vehicle combinations.

In addition, in another example, when the judging module 804 judges whether there is a relationship between the vehicle and the person according to the vehicle identification frame and the person identification frame, it is specifically:

Obtain the coordinate information of the vehicle identification frame and the coordinate information of the personnel identification frame, and obtain the vehicle coordinate information and the personnel coordinate information;

According to the vehicle coordinate information and the personnel coordinate information, it is determined whether there is a relationship between the vehicle and the person.

In addition, in another example, the vehicle coordinate information includes at least the vertex coordinate value L ₁ of the upper left corner of the vehicle identification frame and the vertex coordinate value R ₁ of the lower right corner, and the personnel coordinate information at least includes the vertex coordinate value L ₂ of the upper left corner of the personnel identification frame. and the vertex coordinate value R ₂ of the lower right corner.

Correspondingly, when the judgment module 804 judges whether the vehicle and the person are related according to the vehicle coordinate information and the personnel coordinate information, it is specifically:

Compare the vertex coordinate value L ₁ with the vertex coordinate value L ₂ , and compare the vertex coordinate value R ₁ with the vertex coordinate value R ₂ ;

Determine the positional relationship between the vehicle identification frame and the person identification frame according to the comparison result;

According to the positional relationship, it is judged whether there is a relationship between the vehicle and the person.

In addition, in another example, when the determination module 804 determines the positional relationship between the vehicle identification frame and the person identification frame according to the comparison result, it is specifically:

If the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not less than the vertex coordinate value R ₂ , or, if the vertex coordinate value L ₁ is not less than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an inclusive relationship;

Otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship.

In addition, in another example, after determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship, the judging module 804 is further configured to perform the following operations:

Determine whether the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and whether the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , and whether the vertex coordinate value L ₂ is less than the vertex coordinate value R ₁ ;

If the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , and the vertex coordinate value L ₂ is less than the vertex coordinate value R ₁ , the vehicle identification frame and the person identification frame are determined. The positional relationship of is an overlapping relationship;

Otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.

In addition, in another example, before determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship, the judgment module 804 is further configured to perform the following operations:

According to the vertex coordinate value L ₂ and the vertex coordinate value R ₁ , determine the overlapping area;

Determine the reference area according to the vertex coordinate value L ₁ and the vertex coordinate value R ₂ ;

judging whether the ratio of the overlapping area to the reference area is greater than a first preset threshold;

Correspondingly, if it is greater than, the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship is performed; otherwise, the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship is performed.

In addition, in another example, after determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship, the judging module 804 is further configured to perform the following operations:

Determine whether the vertex coordinate value L ₂ is not greater than the vertex coordinate value L ₁ , and whether the vertex coordinate value R ₂ is not greater than the vertex coordinate value R ₁ , and whether the vertex coordinate value L ₁ is less than the vertex coordinate value R ₂ ;

If the vertex coordinate value L ₂ is not greater than the vertex coordinate value L ₁ , and the vertex coordinate value R ₂ is not greater than the vertex coordinate value R ₁ , and the vertex coordinate value L ₁ is less than the vertex coordinate value R ₂ , the vehicle identification frame and the person identification frame are determined. The positional relationship of is an overlapping relationship;

Otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.

In addition, in another example, before the positional relationship between the fixed vehicle identification frame and the person identification frame is an overlapping relationship, the judgment module 804 is further configured to perform the following operations:

According to the vertex coordinate value R ₂ and the vertex coordinate value L ₁ , determine the overlapping area;

Determine the reference area according to the vertex coordinate value L ₂ and the vertex coordinate value R ₁ ;

judging whether the ratio of the overlapping area to the reference area is greater than a first preset threshold;

Correspondingly, if it is greater than, the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship is performed; otherwise, the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship is performed.

In addition, in another example, when the association module 805 determines whether there is an association between the vehicle and the person according to the positional relationship, it is specifically:

When the location relationship is an inclusive relationship, it is determined that the vehicle and the person are related;

When the positional relationship is an overlapping relationship, it is determined that the vehicle and the person are related;

When the positional relationship is a non-inclusive and non-overlapping relationship, it is determined that there is no association between the vehicle and the person.

In addition, in another example, before determining that there is no association between the vehicle and the person, the judging module 804 is further configured to perform the following operations:

Determine the coordinate value C ₁ of the center point of the vehicle identification frame according to the vertex coordinate value L ₁ and the vertex coordinate value R ₁ ;

Determine the coordinate value C ₂ of the center point of the personnel identification frame according to the vertex coordinate value L ₂ and the vertex coordinate value R ₂ ;

According to the center point coordinate value C ₁ and the center point coordinate value C ₂ , determine the distance D ₁ from the center point of the vehicle identification frame to the center point of the person identification frame;

determining the diagonal distance D ₂ of the video frame image;

Determine whether the ratio of the distance D ₁ to the diagonal distance D ₂ is greater than a second preset threshold;

Correspondingly, if it is greater than that, it is determined that the vehicle and the person are associated; otherwise, the step of determining that the vehicle and the person are not associated is performed.

In addition, in another example, before determining that there is no association between the vehicle and the person, the judgment module 804 is further configured to perform the following operations:

Expand the vehicle identification frame and the person identification frame to obtain the background identification frame;

Perform feature extraction on the background recognition frame, and determine the location information of vehicles and personnel according to the extracted features;

Determine when vehicles and people appear in the video to be processed;

Obtain an associated video associated with the video to be processed, where the associated video is a video captured by a camera located in the same area as the camera that shoots the video to be processed, but located at a different point;

According to the location information and time information, extract the associated video frame image from the associated video provided by each point;

Traverse each associated video frame image, and record the point where the vehicle appears and the point where the person appears;

If the intersection of the point where the vehicle appears and the point where the person appears is greater than the third threshold, it is determined that the vehicle and the person are associated;

Otherwise, the steps of determining that there is no association between the vehicle and the person are performed.

It is not difficult to find that this embodiment is a device embodiment corresponding to the first, or the second, or the third, or the fourth embodiment, and this embodiment may be the same as the first, or the second, or the third, or the The four embodiments are implemented in cooperation with each other. The related technical details mentioned in the first, or the second, or the third, or the fourth embodiment are still valid in this embodiment, and are not repeated here in order to reduce repetition. Correspondingly, the related technical details mentioned in this embodiment can also be applied to the first, or the second, or the third, or the fourth embodiment.

It is worth mentioning that all the modules involved in this embodiment are logical modules. In practical applications, a logical unit may be a physical unit, a part of a physical unit, or multiple physical units. A composite implementation of the unit. In addition, in order to highlight the innovative part of the present application, this embodiment does not introduce units that are not closely related to solving the technical problem raised by the present application, but this does not mean that there are no other units in this embodiment.

The sixth embodiment of the present application relates to a person-vehicle information association device, as shown in FIG. 9 , comprising: at least one processor 901 ; and a memory 902 communicatively connected to the at least one processor 901 ; wherein, the memory 902 stores There are instructions executable by the at least one processor 901, and the instructions are executed by the at least one processor 901, so that the at least one processor 901 can execute the method for associating person-vehicle information described in the above method embodiments.

The memory 902 and the processor 901 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 901 and various circuits of the memory 902 together. The bus may also connect together various other circuits, such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides the interface between the bus and the transceiver. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. The data processed by the processor 901 is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor 901 .

Processor 901 is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 902 may be used to store data used by the processor 901 when performing operations.

A seventh embodiment of the present application relates to a computer-readable storage medium storing a computer program. When the computer program is executed by the processor, the method for associating person-vehicle information described in the above method embodiments is implemented.

That is, those skilled in the art can understand that all or part of the steps in the method of implementing the above embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium and includes several instructions to make a device ( It may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present application, and in practical applications, various changes in form and details can be made without departing from the spirit and the spirit of the present application. Scope.

Claims (16)

1. A human-vehicle information association method, comprising:

   Get the pending video;

   Identify vehicles and people included in the video to be processed, and obtain a vehicle identification frame and a person identification frame;

   Perform feature extraction on the vehicle identification frame and the person identification frame respectively to obtain vehicle information and personnel information;

   According to the vehicle identification frame and the person identification frame, determine whether the vehicle and the person are associated;

   If there is an association, the vehicle information and the person information are associated.
2. The method for associating human-vehicle information as claimed in claim 1, wherein identifying vehicles and persons included in the video to be processed to obtain a vehicle identification frame and a person identification frame, comprising:

   Extract video frame images from the video to be processed;

   Perform vehicle detection on the video frame image, and when a vehicle is detected, demarcate all the vehicles appearing in the video frame image in the video frame image, and obtain N vehicle identification frames, where N is an integer greater than 0;

   Perform face and human shape detection on the video frame image, and when a person is detected, all personnel appearing in the video frame image are demarcated in the video frame image, and M personnel identification frames are obtained, where M is greater than 0. Integer.
3. The method for associating human-vehicle information according to claim 2, wherein, before judging whether the vehicle and the person are associated according to the vehicle identification frame and the person identification frame, the method further comprises:

   Combining N vehicle identification frames and M personnel identification frames to obtain N×M personal vehicle combinations;

   Traverse N×M personal-vehicle combinations, and for each traversed personal-vehicle combination, perform the step of judging whether the vehicle and the person are associated according to the vehicle identification frame and the person identification frame.
4. The method for associating person-vehicle information according to any one of claims 1 to 3, wherein the determining whether the vehicle and the person are associated according to the vehicle identification frame and the person identification frame includes:

   Obtain the coordinate information of the vehicle identification frame and the coordinate information of the personnel identification frame, and obtain the vehicle coordinate information and the personnel coordinate information;

   According to the vehicle coordinate information and the person coordinate information, it is determined whether the vehicle and the person are related.
5. The method for associating human-vehicle information according to claim 4, wherein the vehicle coordinate information at least includes a vertex coordinate value L ₁ at the upper left corner of the vehicle identification frame and a vertex coordinate value R ₁ at the lower right corner, and the personnel coordinate information at least includes The vertex coordinate value L ₂ of the upper left corner of the personnel identification frame and the vertex coordinate value R ₂ of the lower right corner;

   The determining whether the vehicle and the person are related according to the vehicle coordinate information and the person coordinate information includes:

   comparing the vertex coordinate value L ₁ with the vertex coordinate value L ₂ , and comparing the vertex coordinate value R ₁ with the vertex coordinate value R ₂ ;

   Determine the positional relationship between the vehicle identification frame and the person identification frame according to the comparison result;

   According to the positional relationship, it is determined whether the vehicle and the person are related.
6. The method for associating person and vehicle information according to claim 5, wherein the determining the positional relationship between the vehicle identification frame and the person identification frame according to the comparison result comprises:

   If the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not less than the vertex coordinate value R ₂ , or if the vertex coordinate value L ₁ is not less than the vertex coordinate value The vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , then it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an inclusive relationship;

   Otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship.
7. The method for associating person-vehicle information according to claim 6, wherein after the determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship, the method further comprises:

   Determine whether the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and whether the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , and whether the vertex coordinate value L ₂ is smaller than the vertex coordinate value vertex coordinate value R ₁ ;

   If the vertex coordinate value L ₁ is not greater than the vertex coordinate value L ₂ , and the vertex coordinate value R ₁ is not greater than the vertex coordinate value R ₂ , and the vertex coordinate value L ₂ is less than the vertex coordinate value R ₁ , then it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship;

   Otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.
8. The method for associating human-vehicle information according to claim 7, wherein before the determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship, the method further comprises:

   According to the vertex coordinate value L ₂ and the vertex coordinate value R ₁ , determine the overlapping area;

   Determine the reference area according to the vertex coordinate value L ₁ and the vertex coordinate value R ₂ ;

   judging whether the ratio of the overlapping area to the reference area is greater than a first preset threshold;

   If it is greater than that, execute the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship;

   Otherwise, the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship is performed.
9. The method for associating person-vehicle information according to claim 6, wherein after the determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive relationship, the method further comprises:

   Determine whether the vertex coordinate value L ₂ is not greater than the vertex coordinate value L ₁ , and whether the vertex coordinate value R ₂ is not greater than the vertex coordinate value R ₁ , and whether the vertex coordinate value L ₁ is smaller than the vertex coordinate value R 1 . vertex coordinate value R ₂ ;

   If the vertex coordinate value L ₂ is not greater than the vertex coordinate value L ₁ , and the vertex coordinate value R ₂ is not greater than the vertex coordinate value R ₁ , and the vertex coordinate value L ₁ is less than the vertex coordinate value R ₂ , then it is determined that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship;

   Otherwise, it is determined that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship.
10. The method for associating person-vehicle information according to claim 9, wherein before the determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship, the method further comprises:

    According to the vertex coordinate value R ₂ and the vertex coordinate value L ₁ , determine the overlapping area;

    According to the vertex coordinate value L ₂ and the vertex coordinate value R ₁ , determine the reference area;

    judging whether the ratio of the overlapping area to the reference area is greater than a first preset threshold;

    If it is greater than that, execute the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is an overlapping relationship;

    Otherwise, the step of determining that the positional relationship between the vehicle identification frame and the person identification frame is a non-inclusive and non-overlapping relationship is performed.
11. The method for associating person-vehicle information according to any one of claims 5 to 10, wherein the determining whether the vehicle and the person are associated according to the positional relationship includes:

    When the location relationship is an inclusive relationship, determining that the vehicle and the person are associated;

    When the positional relationship is an overlapping relationship, determining that the vehicle and the person are associated;

    When the positional relationship is a non-inclusive and non-overlapping relationship, it is determined that the vehicle and the person are not associated.
12. The method for associating person-vehicle information according to claim 11, wherein before the determining that the vehicle and the person are not associated, the method further comprises:

    According to the vertex coordinate value L ₁ and the vertex coordinate value R ₁ , determine the center point coordinate value C ₁ of the vehicle identification frame;

    According to the vertex coordinate value L ₂ and the vertex coordinate value R ₂ , determine the center point coordinate value C ₂ of the person identification frame;

    According to the coordinate value C ₁ of the center point and the coordinate value C ₂ of the center point, determine the distance D ₁ from the center point of the vehicle identification frame to the center point of the person identification frame;

    determining the diagonal distance D ₂ of the video frame image;

    Determine whether the _ratio of the distance D1 to the diagonal distance D2 is greater than a _second preset threshold;

    If it is greater than, it is determined that the vehicle and the person are associated;

    Otherwise, the step of determining that there is no association between the vehicle and the person is performed.
13. The method for associating person-vehicle information according to claim 12, wherein before the determining that the vehicle and the person are not associated, the method further comprises:

    Enlarging the vehicle identification frame and the person identification frame to obtain a background identification frame;

    Perform feature extraction on the background recognition frame, and determine the location information of the vehicle and the person according to the extracted features;

    determining the time information of the appearance of the vehicle and the person in the video to be processed;

    Acquire an associated video associated with the video to be processed, where the associated video is a video captured by a camera located in the same area as the camera that shoots the video to be processed, but located at a different point;

    According to the location information and the time information, extract the associated video frame image from the associated video provided by each point;

    Traverse each associated video frame image, and record the point where the vehicle appears and the point where the person appears;

    If the intersection of the point where the vehicle appears and the point where the person appears is greater than a third threshold, it is determined that the vehicle and the person are associated;

    Otherwise, the step of determining that there is no association between the vehicle and the person is performed.
14. A person and vehicle information association device, comprising:

    The acquisition module is used to acquire the video to be processed;

    an identification module for identifying vehicles and persons included in the to-be-processed video to obtain a vehicle identification frame and a person identification frame;

    an extraction module, configured to perform feature extraction on the vehicle identification frame and the person identification frame respectively to obtain vehicle information and personnel information;

    a judgment module for judging whether the vehicle and the person are related according to the vehicle identification frame and the person identification frame;

    an association module, configured to associate the vehicle information with the personnel information when the vehicle and the person are associated.
15. A human-vehicle information association device, comprising:

    at least one processor; and,

    a memory communicatively coupled to the at least one processor; wherein,

    the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any one of claims 1 to 13 The method of association of people and vehicles information.
16. A computer-readable storage medium storing a computer program, when the computer program is executed by a processor, the method for associating person-vehicle information according to any one of claims 1 to 13 is implemented.

Images