CN117292327A - Method, device, equipment and medium for associating targets - Google Patents
Method, device, equipment and medium for associating targets Download PDFInfo
- Publication number
- CN117292327A CN117292327A CN202311573795.0A CN202311573795A CN117292327A CN 117292327 A CN117292327 A CN 117292327A CN 202311573795 A CN202311573795 A CN 202311573795A CN 117292327 A CN117292327 A CN 117292327A
- Authority
- CN
- China
- Prior art keywords
- target
- person
- candidate persons
- obtaining
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012544 monitoring process Methods 0.000 claims abstract description 70
- 238000001514 detection method Methods 0.000 claims description 16
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 238000007689 inspection Methods 0.000 abstract description 61
- 230000008569 process Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 5
- WVAKRQOMAINQPU-UHFFFAOYSA-N 2-[4-[2-[5-(2,2-dimethylbutyl)-1h-imidazol-2-yl]ethyl]phenyl]pyridine Chemical compound N1C(CC(C)(C)CC)=CN=C1CCC1=CC=C(C=2N=CC=CC=2)C=C1 WVAKRQOMAINQPU-UHFFFAOYSA-N 0.000 description 4
- OENIXTHWZWFYIV-UHFFFAOYSA-N 2-[4-[2-[5-(cyclopentylmethyl)-1h-imidazol-2-yl]ethyl]phenyl]benzoic acid Chemical compound OC(=O)C1=CC=CC=C1C(C=C1)=CC=C1CCC(N1)=NC=C1CC1CCCC1 OENIXTHWZWFYIV-UHFFFAOYSA-N 0.000 description 4
- 101150033765 BAG1 gene Proteins 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 101100111184 Schizosaccharomyces pombe (strain 972 / ATCC 24843) bag101 gene Proteins 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 150000002739 metals Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device, equipment and a medium for associating targets. The association method of the targets comprises the following steps: obtaining a monitoring image, wherein the monitoring image comprises a target and a plurality of candidate persons; identifying a monitoring image to obtain identification results of a target and a plurality of candidate persons; obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the identification results of the target and the plurality of candidate persons; and determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons. By adopting the method and the device, the owner of the forbidden package can be rapidly and accurately determined under the condition that the security inspection flow and the security inspection efficiency are not affected according to the monitoring image of the pedestrian (personnel) when the package (target) is placed into the security inspection machine, so that the security inspection personnel can spend a large amount of time and energy to look back for the monitoring to find out the carrier of the forbidden package, and further, the security inspection efficiency is greatly improved.
Description
Technical Field
The present disclosure relates generally to the field of computer vision, and in particular, to the field of object detection, and in particular, to a method, apparatus, device, and medium for associating objects.
Background
At present, when public transportation such as high-speed rail, train and the like goes out, security inspection is needed through a security inspection channel, and the purpose of security inspection is mainly to prevent travel staff from carrying in banners such as firearms, control cutters, inflammable and explosive objects and the like, however, as various portable objects carried by the travel staff and the flow of the travel staff are large, once the condition of missing inspection occurs, a great deal of time is needed to check the owners carrying the banners by checking and monitoring playback.
In addition, in the security inspection process, the packages are generally arranged in disorder and overlapped and staggered, once dangerous objects are detected, the owners carrying the dangerous objects possibly perceive abnormality, the packages are abandoned to leave or not acknowledged as all the packages, the tracing work is difficult, and a great deal of time is required to be consumed for checking, monitoring, playback and the like, so that the security inspection efficiency is low.
Disclosure of Invention
Based on the above, it is necessary to provide a method, a device, an apparatus and a medium for associating objects, which can quickly and accurately determine the owner of the forbidden package without affecting the security process and the security efficiency according to the monitoring image when the package (object) is put into the security inspection machine by pedestrians (personnel), thereby avoiding the security inspection personnel spending a great deal of time and effort to look back for monitoring to find out the carrier of the forbidden package, and further greatly improving the security inspection efficiency.
In a first aspect, there is provided a method of associating objects, the method comprising:
obtaining a monitoring image, wherein the monitoring image comprises a target and a plurality of candidate persons;
identifying the monitoring image, and obtaining identification results of the target and a plurality of candidate persons;
obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the identification results of the target and the plurality of candidate persons;
and determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
Further, the identifying the monitoring image, obtaining the identification result of the target and the plurality of candidate persons, includes:
cutting out a target area image from the monitoring image, wherein the target area image comprises the target and a plurality of candidate persons;
extracting features of the target area image to obtain a global feature image;
and inputting the global characteristic image into a detection model to obtain the identification frames and the positions of the identification frames of the target and the plurality of candidate persons.
Further, the step of obtaining the intersection ratio between the target and each of the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons includes:
obtaining the intersection area of the intersection part of the identification frame of the target and the identification frame of the first person according to the position of the identification frame of the target and the position of the identification frame of the first person, wherein the first person is any person in the plurality of candidate persons;
and obtaining the intersection ratio between the target and the first person according to the ratio between the intersection area and the area of the identification frame of the target.
Further, the obtaining a distance value between the target and each of the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons includes:
judging whether an intersection part exists between an identification frame of the target and an identification frame of a second person, wherein the second person is any person in the plurality of candidate persons;
if so, obtaining a center distance between the center point of the identification frame of the target and the center point of the identification frame of the second person;
obtaining a frame size of an identification frame of the target and a frame size of an identification frame of the second person;
and obtaining a distance value between the target and the second person according to the center distance, the frame size of the identification frame of the target and the frame size of the identification frame of the second person.
Further, the distance value between the target and the second person is achieved by the following formula:
and distance is the distance value, d1 is the center distance, wp and Hp are the width and the height of the identification frame of the second person respectively, and Wb and Hb are the width and the height of the identification frame of the target respectively.
Further, in the case that it is judged that there is no intersection between the identification frame of the target and the identification frame of the second person, the distance value between the target and the second person is set to a default value, wherein the default value is 0.
Further, the determining, from the candidate persons, the person associated with the target according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons includes:
obtaining the weight ratio of the intersection ratio and the distance value in the calculation of the personnel associated with the target;
obtaining a cost value between the target and each person according to the weight ratio, the intersection ratio and the distance value between the target and each person;
obtaining the largest cost value among the cost values between the target and each person;
and determining the personnel associated with the target according to the maximum cost value.
In a second aspect, there is provided an association apparatus for an object, the apparatus comprising:
the device comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a monitoring image, and the monitoring image comprises a target and a plurality of candidate persons;
the identification module is used for identifying the monitoring image and obtaining identification results of the target and the plurality of candidate persons;
the computing module is used for obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons;
and the association module is used for determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
In a third aspect, a computer device is provided, comprising a memory, a processor and a computer program stored on no memory and executable on the processor, characterized in that the processor implements the steps of the method of the first aspect and any one of the possible implementations of the first aspect when the program is executed by the processor.
In a fourth aspect, a computer readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, implements the steps of the method of the first aspect and any one of the possible implementations of the first aspect.
By adopting the embodiment of the application, the monitoring image of the pedestrian (person) when the package (target) is put into the security inspection machine is obtained, the image is identified by the target and the person, when the target and a plurality of candidate persons are identified, the intersection ratio and the distance value between the target and each person are obtained according to the identification result, and finally, the person associated with the target is determined according to the intersection ratio and the distance value between the target and each person, namely: and determining packages carried by different persons. The method and the system can quickly and accurately determine the corresponding relation between the package and the passenger (which passenger the package is carried) under the condition that the security inspection flow and the security inspection efficiency are not affected, so that when suspicious objects are in the package, the suspicious objects can be quickly and accurately traced back to video playback of the security inspection process, the efficiency of finding the owner of the package is effectively improved, and the security inspection efficiency is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings, in which:
FIG. 1 is a schematic diagram of a correlation system for targets provided by embodiments of the present application;
FIG. 2 is a flow chart of a method for associating objects provided in an embodiment of the present application;
FIG. 3 is another flow chart of a method for associating objects according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of identifying personnel and packages in a method for associating objects provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of an association device of an object provided in an embodiment of the present application;
fig. 6 is a block diagram of a computer device according to an embodiment of the present application.
Detailed Description
The present application is described in further detail below with reference to examples and figures. It is to be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to be limiting. It should be noted that, for convenience of description, only the portions related to the application are shown in the drawings.
It should be noted that, without conflict, the embodiments herein, i.e., features of the embodiments, may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
At present, when an X-ray security inspection machine identifies forbidden packages, security inspection personnel are required to look back at a monitoring video so as to determine natural light images of the forbidden packages, and the security inspection personnel can identify face information in the natural light images of the forbidden packages through human eyes, so that passengers carrying the forbidden packages are searched. However, during security inspection, the articles (such as packages) placed in the conveyor belt of the security inspection machine are randomly placed and disordered, and when the traffic of people is large, the placed articles may overlap and stagger, once dangerous articles are detected, the passengers carrying the dangerous articles may notice abnormality, discard the packages or do not acknowledge the packages, at this time, security inspection personnel need to consume a great deal of time and effort, and the specific passengers carrying the forbidden packages are determined by monitoring video playback, so that tracing work is difficult and security inspection efficiency is low. To solve the problem of difficult tracing, although there are some places with different case flows, for example: more security inspection channels are opened, and security inspection processes are carried out on each passenger and articles carried by the passengers one by one in each security inspection channel, however, the security inspection mode requires more manpower and material resource cost, and the participation degree of the passengers is high, so that the security inspection speed is low, the experience is poor, and the security inspection efficiency is limited.
Based on the above, the related method, device, equipment and storage medium of the target can rapidly and accurately determine the owner of the forbidden package under the condition that the security flow and the security efficiency are not affected according to the monitoring image when the package (target) is put into the security inspection machine by pedestrians (personnel), so that the security inspection personnel can spend a great deal of time and effort to look back for monitoring to find out the carrier of the forbidden package, and the security inspection efficiency is greatly improved.
The method for associating the targets can be applied to a system for associating the targets shown in fig. 1. Referring to fig. 1, the association system of the object may include an X-ray security inspection machine 10, a camera 20, and a computer device 30. The X-ray security inspection machine 10 may record X-ray images of packages, for example, illicit package X-ray images. The camera 20 may capture a monitoring image of each passenger during placement of the package into the conveyor of the X-ray security inspection machine 10. The computer device 30 may obtain a monitoring image including a target and a plurality of candidate persons; identifying a monitoring image to obtain identification results of a target and a plurality of candidate persons; obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the identification results of the target and the plurality of candidate persons; and determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons. The method can quickly and accurately determine the owner of the forbidden package under the condition that the security inspection process and the security inspection efficiency are not affected according to the monitoring image of the pedestrian (personnel) when the package (target) is placed in the security inspection machine, so that the security inspection personnel can spend a great deal of time and effort to review the monitoring to find out the carrier of the forbidden package, and the security inspection efficiency is greatly improved.
When the X-rays penetrate through different parts of the package, the absorption degree of the different parts on the X-rays is different, and different images can be obtained after development treatment, for example, organic matters such as food, plastics and the like are displayed as orange on a screen of the security inspection machine, books, ceramics and the like are displayed as green on the screen, and metals are displayed as blue.
The camera 20 may record a monitoring video of the process of placing the package by the passenger, and may also capture a monitoring image of the process of placing the package by the customer. For example, the monitoring image of the customer placement package process may be taken right up to down or diagonally down, for example: the camera 20 is mounted above the X-ray security inspection machine 10 so that the photographing angle is directed against the passenger and the articles they carry.
The computer device 30 may be an independent server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (content delivery network, CDN), basic cloud computing services such as big data and artificial intelligence platforms, and the like.
Fig. 1 above presents an application scenario diagram of the present application. In one embodiment of the present application, a method of associating objects is provided that may be applied to the computer device 30 shown in FIG. 1. As shown in fig. 2, the method comprises the steps of:
s201: a monitoring image is obtained, wherein the monitoring image comprises a target and a plurality of candidate persons.
As shown in connection with fig. 1, the monitoring image may be captured by the camera 20. Specifically, the camera 20 is installed above the security inspection machine 10, and the photographing angle of the camera 20 may be the direction facing the passenger putting the article (package) on the conveyor belt of the security inspection machine 10, so that the process of putting the package per passenger can be clearly photographed. The camera 20 captures the movement of the passenger to place the package as the passenger places the package in the conveyor of the security check machine 10, and the captured monitoring image includes a clear image of the passenger and the package.
In the process of security inspection, the flow of people is large, and there is often a process of placing a plurality of passengers into respective packages in the conveyor belt of the security inspection machine 10 in a short period of time, so that there are often a plurality of passengers and a plurality of packages in one monitoring image. In the embodiment of the application, a plurality of passengers appearing in one monitoring image are called a plurality of candidate persons, and the target is any one of a plurality of packages in the monitoring image.
The monitoring image may be a picture that is taken when a passenger puts in a package, or may be a monitoring video that is taken during the process of putting in a package by the passenger, then a key frame is obtained from the monitoring video, and an image of the key frame is used as the monitoring image, where the key frame may be a corresponding frame of image in the monitoring video when the passenger just puts in the package in the conveyor belt of the security inspection machine 10 in the monitoring video.
S202: and identifying the monitoring image to obtain identification results of the target and the plurality of candidate persons.
The image recognition mode can be to use a pre-trained target detection model (simply referred to as a detection model), wherein the target detection model takes a global feature image of a monitoring image as input, and the output is personnel and articles (namely, packages) in the monitoring image.
Specifically, as shown in fig. 3, identifying the monitoring image, obtaining the identification result of the target and the plurality of candidate persons includes:
s2021: a target area image is cropped from the monitoring image, wherein the target area image includes a target and a plurality of candidate persons. Namely: a region of interest (ROI, region of intrest) is extracted from the monitored image, and then the region of interest is cropped out as a target region image according to the position of the region of interest in the image, wherein the region of interest refers to a region including personnel and packages.
The monitoring image is cut, unnecessary areas are removed, noise in the image can be effectively reduced, the subsequent calculation amount for image processing is reduced, and the image can be better processed.
S2022: and extracting the characteristics of the target area image to obtain a global characteristic image.
Namely: global features are extracted from the target area image, and a global feature image can be obtained, wherein the global features are integral attributes of the image, including color, texture and shape features, such as intensity histograms. The global features have the characteristics of good invariance, simple calculation, visual representation and the like.
In a specific example, global feature extraction of an image may be performed by an encoder, namely: and inputting the target area image into an encoder to obtain the global characteristic of the target area image.
S2023: and inputting the global characteristic image into a detection model to obtain the identification frames of the target and the plurality of candidate persons and the positions of the identification frames.
As shown in fig. 4, the recognition frames of the targets and the persons recognized by the detection model and the positions of the recognition frames are shown. Where Person1 represents the identification box of the identified Person, bag1 represents the identified one object (i.e., one package), and Bag2 represents the identified other object (i.e., another package). As can be seen from fig. 4, the Person on1 identifies the positional relationship between the frame and Bag1 and Bag2 packages.
The detection model is trained in advance, global characteristic images of the monitoring images are taken as input, and the output is information such as identification frames of people and objects in the monitoring images. In one embodiment of the present application, the training process of the detection model may be performed as follows:
obtaining an image sample, such as a large number of pre-acquired monitoring images, then taking a recognition frame of a person in the image sample and a recognition frame of a package as labels, inputting the image sample into an initial detection model, training the initial detection model according to the output of the initial detection model and the loss between the labels, and further obtaining a trained detection model.
S203: and obtaining the intersection ratio and the distance value between the target and each person in the plurality of candidate persons according to the identification results of the target and the plurality of candidate persons.
In a specific example, obtaining the intersection ratio between the target and each of the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons includes: obtaining the intersection area of the intersection part of the identification frame of the target and the identification frame of the first person according to the position of the identification frame of the target and the position of the identification frame of the first person, wherein the first person is any person in the plurality of candidate persons; and obtaining the intersection ratio between the target and the first person according to the ratio between the intersection area and the area of the identification frame of the target.
Referring to fig. 4, the ratio iou=cr/Br between Bag1 and Person1, where IOU represents the ratio, cr represents the area of the portion where the Person1 and Bag1 are intersected, and Br represents the total area of Bag 1. Similarly, the intersection ratio iou=cr/Br between Bag2 and Person1, where iou=cr/Br represents the intersection ratio, cr represents the area of the portion where the recognition frame of Person1 and the recognition frame of Bag2 intersect, and Br represents the total area of the recognition frames of Bag 2.
Specifically, the area of the identification frame of the passenger Person1 is denoted as Pr, the area of the identification frame of the package is denoted as Br, the area of the intersecting part of Pr and Br is denoted as Cr, the intersection ratio between the two frames is iou=cr/Br, it can be understood that the higher the overlapping degree of the package and the passenger is, the larger the intersection ratio is, and the maximum value is 1, namely: the package is completely in the pedestrian detection frame, wherein the calculation mode of the IOU is shown in formula 1:
（1）
where IOU, i.e., IOU, represents the cross-over ratio.
The above example describes how the process of obtaining the intersection ratio between the target and each of the plurality of candidate persons is based on the recognition results of the target and the plurality of candidate persons, and how the distance value between the target and each of the plurality of candidate persons is obtained based on the recognition results of the target and the plurality of candidate persons will be described below in connection with specific examples.
Specifically, according to the recognition results of the target and the plurality of candidate persons, obtaining a distance value between the target and each of the plurality of candidate persons includes: judging whether an intersection part exists between an identification frame of the target and an identification frame of a second person, wherein the second person is any person in the plurality of candidate persons; if so, obtaining a center distance between the center point of the identification frame of the target and the center point of the identification frame of the second person; obtaining a frame size of an identification frame of the target and a frame size of an identification frame of the second person; and obtaining a distance value between the target and the second person according to the center distance, the frame size of the identification frame of the target and the frame size of the identification frame of the second person.
In this example, the distance value between the target and the second person is achieved by the following formula 2, the formula 2 being:
（2）
wherein distance is a distance value, d1 is a center distance, wp and Hp are the width and the height of the identification frame of the second person, respectively, and Wb and Hb are the width and the height of the identification frame of the target, respectively.
Namely: taking the distance from the center point of the identification frame of the passenger to the center point of the identification frame of the package as d1, the width and height of the identification frame of the passenger as Wp and Hp respectively, and the width and height of the identification frame of the package as Wb and Hb respectively, and calculating the distance value between the passenger and the package through the formula 2.
It will be appreciated that if there is an intersection of the identification frame of the passenger and the identification frame of the package, a greater distance between the center points will indicate a lower match value, and conversely, a smaller distance between the center points will indicate a higher match value. Wherein, the higher the matching value, the greater the degree of characterizing the target and including the association, and conversely, the lesser the degree of the target and including the association.
Further, if the passenger's identification frame and the package's identification frame have no intersecting portion, distance is noted as 0, for example: in the case where it is judged that there is no intersection between the identification frame of the target and the identification frame of the second person, the distance value between the target and the second person is set to a default value, that is: set to 0, indicating that the target is associated with a very small degree of second person.
S204: and determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
In one specific example, determining a person associated with the target from among the candidate persons based on an intersection ratio and a distance value between the target and each person in the plurality of candidate persons, includes: obtaining the weight ratio of the intersection ratio and the distance value in the calculation of the personnel associated with the target; obtaining a cost value between the target and each person according to the weight ratio, the intersection ratio and the distance value between the target and each person; obtaining the largest cost value among the cost values between the target and each person; and determining the personnel associated with the target according to the maximum cost value.
Specifically, a cost matrix of the package and the passenger is established, which relates to distance calculation and the calculation of the intersection ratio IOU, and the combination of the two calculates the cost matrix to find out the optimal matching of the package and the pedestrian (namely, the package and the passenger with the highest association degree), wherein: the calculation formula of the cost matrix is shown in formula 3:
（3）
the cost represents the cost value of the passengers and the packages, and the larger the cost value is, the higher the matching rate of the passengers and the packages is (namely, the higher the association degree is); otherwise, the lower the matching relationship between the passenger and the package.
In formula 3, calcualteiou represents calculating the IOU values of the passenger and the package, the specific calculation mode is shown in formula 1, calcualtedistancesimill represents calculating the distance values of the passenger and the package, and the specific calculation mode is shown in formula 2; the kIOU represents the weight ratio of the IOU value and the distance value when the cost value is calculated, and the default value can be 0.6.
Bringing equations 1 and 2 into equation 3 yields equation 4:
（4）
by equation 4, the cost value between the target and each person can be calculated. For example: calculating cost values between the target and the personnel 1, the personnel 2 and the personnel 3 respectively as a cost value 1, a cost value 2 and a cost value 3, and assuming that the cost value 1 is smaller than the cost value 2 and the cost value 2 is smaller than the cost value 3, namely: the cost value 3 is the largest, and the cost value 3 corresponds to the person 3, and it is finally determined that the target and the person 3 have association, for example: the object is a package, which indicates that the package is an item carried by person 3, and further, the items carried by different persons are rapidly determined.
Based on the above steps S201 to S204, in a specific application scenario, after the pedestrian (passenger) enters the security inspection process: the camera starts shooting images of the process of placing packages on the conveyor belt by pedestrians; acquiring a monitoring image shot by a camera, sending the monitoring image into a target detection model, and acquiring identification frames of pedestrians and packages and positions of the identification frames; establishing a mapping relation between the package and the pedestrian on the package, calculating a cost value between the package and the pedestrian based on a cost function, carrying out blood drainage on the cost value, finding out the package and the pedestrian with the largest cost value, and associating the package and the pedestrian, namely: it is determined that the package is entered by the pedestrian.
According to the method for associating the targets, which is disclosed by the embodiment of the application, the monitoring image of the pedestrian (person) when the package (target) is put into the security inspection machine is obtained, the image is identified by the target and the person, when the target and a plurality of candidate persons are identified, the intersection ratio and the distance value between the target and each person are obtained according to the identification result, and finally, the person associated with the target is determined according to the intersection ratio and the distance value between the target and each person, namely: and determining packages carried by different persons. The method and the system can quickly and accurately determine the corresponding relation between the package and the passenger (which passenger the package is carried) under the condition that the security inspection flow and the security inspection efficiency are not affected, so that when suspicious objects are in the package, the suspicious objects can be quickly and accurately traced back to video playback of the security inspection process, the efficiency of finding the owner of the package is effectively improved, and the security inspection efficiency is improved.
As shown in fig. 5, an embodiment of the present application provides an association apparatus of an object, including: an acquisition module 510, an identification module 520, a calculation module 530, and an association module 540, wherein:
an acquisition module 510 for acquiring a monitoring image, wherein the monitoring image includes a target and a plurality of candidate persons;
the identifying module 520 is configured to identify the monitoring image, and obtain identification results of the target and the plurality of candidate persons;
a calculating module 530, configured to obtain an intersection ratio and a distance value between the target and each of the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons;
and the association module 540 is used for determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
According to the target association device in the embodiment of the application, a monitoring image of a pedestrian (person) when a package (target) is placed in a security inspection machine is obtained, the image is identified by the target and the person, when the target and a plurality of candidate persons are identified, an intersection ratio and a distance value between the target and each person are obtained according to the identification result, and finally, the person associated with the target is determined according to the intersection ratio and the distance value between the target and each person, namely: and determining packages carried by different persons. The method and the system can quickly and accurately determine the corresponding relation between the package and the passenger (which passenger the package is carried) under the condition that the security inspection flow and the security inspection efficiency are not affected, so that when suspicious objects are in the package, the suspicious objects can be quickly and accurately traced back to video playback of the security inspection process, the efficiency of finding the owner of the package is effectively improved, and the security inspection efficiency is improved.
For specific limitations on the association means of the object, reference may be made to the above limitation on the association method of the object, and details are not repeated here. The various modules of the associated apparatus of the above objects may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory of the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, as shown in FIG. 6, a computer device is provided, including but not limited to an internal memory having a computer program stored therein and a processor that when executed performs the computer program:
obtaining a monitoring image, wherein the monitoring image comprises a target and a plurality of candidate persons;
identifying the monitoring image, and obtaining identification results of the target and a plurality of candidate persons;
obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the identification results of the target and the plurality of candidate persons;
and determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
Embodiments of the present application provide a computer program product comprising instructions which, when executed, cause a method as described in embodiments of the present application to be performed. For example, various steps of the association method of the targets shown in fig. 2 may be performed.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the claims. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. A method of associating objects, the method comprising:
obtaining a monitoring image, wherein the monitoring image comprises a target and a plurality of candidate persons;
identifying the monitoring image, and obtaining identification results of the target and a plurality of candidate persons;
obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the identification results of the target and the plurality of candidate persons;
and determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
2. The method according to claim 1, wherein the identifying the monitoring image to obtain the identification result of the target and the plurality of candidate persons includes:
cutting out a target area image from the monitoring image, wherein the target area image comprises the target and a plurality of candidate persons;
extracting features of the target area image to obtain a global feature image;
and inputting the global characteristic image into a detection model to obtain the identification frames and the positions of the identification frames of the target and the plurality of candidate persons.
3. The method according to claim 2, wherein the obtaining the intersection ratio between the target and each of the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons comprises:
obtaining the intersection area of the intersection part of the identification frame of the target and the identification frame of the first person according to the position of the identification frame of the target and the position of the identification frame of the first person, wherein the first person is any person in the plurality of candidate persons;
and obtaining the intersection ratio between the target and the first person according to the ratio between the intersection area and the area of the identification frame of the target.
4. The method according to claim 2, wherein the obtaining a distance value between the target and each of the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons includes:
judging whether an intersection part exists between an identification frame of the target and an identification frame of a second person, wherein the second person is any person in the plurality of candidate persons;
if so, obtaining a center distance between the center point of the identification frame of the target and the center point of the identification frame of the second person;
obtaining a frame size of an identification frame of the target and a frame size of an identification frame of the second person;
and obtaining a distance value between the target and the second person according to the center distance, the frame size of the identification frame of the target and the frame size of the identification frame of the second person.
5. The method of claim 4, wherein the distance value between the target and the second person is achieved by the following formula:
and distance is the distance value, d1 is the center distance, wp and Hp are the width and the height of the identification frame of the second person respectively, and Wb and Hb are the width and the height of the identification frame of the target respectively.
6. The method according to claim 4, wherein in a case where it is determined that there is no intersection between the identification frame of the object and the identification frame of the second person, a distance value between the object and the second person is set to a default value, wherein the default value is 0.
7. The method according to any one of claims 1 to 6, wherein the determining, from the candidate persons, the person associated with the target according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons includes:
obtaining the weight ratio of the intersection ratio and the distance value in the calculation of the personnel associated with the target;
obtaining a cost value between the target and each person according to the weight ratio, the intersection ratio and the distance value between the target and each person;
obtaining the largest cost value among the cost values between the target and each person;
and determining the personnel associated with the target according to the maximum cost value.
8. An apparatus for associating objects, comprising:
the device comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a monitoring image, and the monitoring image comprises a target and a plurality of candidate persons;
the identification module is used for identifying the monitoring image and obtaining identification results of the target and the plurality of candidate persons;
the computing module is used for obtaining the intersection ratio and the distance value between each person in the target and the plurality of candidate persons according to the recognition results of the target and the plurality of candidate persons;
and the association module is used for determining the person associated with the target from the candidate persons according to the intersection ratio and the distance value between the target and each person in the plurality of candidate persons.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the association method of the object according to any of the claims 1-7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the association method of the object according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311573795.0A CN117292327A (en) | 2023-11-23 | 2023-11-23 | Method, device, equipment and medium for associating targets |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311573795.0A CN117292327A (en) | 2023-11-23 | 2023-11-23 | Method, device, equipment and medium for associating targets |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117292327A true CN117292327A (en) | 2023-12-26 |
Family
ID=89239323
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311573795.0A Pending CN117292327A (en) | 2023-11-23 | 2023-11-23 | Method, device, equipment and medium for associating targets |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117292327A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110288627A (en) * | 2019-05-22 | 2019-09-27 | 江苏大学 | One kind being based on deep learning and the associated online multi-object tracking method of data |
| CN112507786A (en) * | 2020-11-03 | 2021-03-16 | 浙江大华技术股份有限公司 | Human body multi-part detection frame association method and device, electronic device and storage medium |
| CN112699748A (en) * | 2020-12-21 | 2021-04-23 | 南京理工大学 | Human-vehicle distance estimation method based on YOLO and RGB image |
| CN114445769A (en) * | 2021-12-31 | 2022-05-06 | 浙江大华技术股份有限公司 | Fishing behavior detection method, device and system |
-
2023
- 2023-11-23 CN CN202311573795.0A patent/CN117292327A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110288627A (en) * | 2019-05-22 | 2019-09-27 | 江苏大学 | One kind being based on deep learning and the associated online multi-object tracking method of data |
| CN112507786A (en) * | 2020-11-03 | 2021-03-16 | 浙江大华技术股份有限公司 | Human body multi-part detection frame association method and device, electronic device and storage medium |
| CN112699748A (en) * | 2020-12-21 | 2021-04-23 | 南京理工大学 | Human-vehicle distance estimation method based on YOLO and RGB image |
| CN114445769A (en) * | 2021-12-31 | 2022-05-06 | 浙江大华技术股份有限公司 | Fishing behavior detection method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10762338B2 (en) | Method and apparatus for detecting fake license plates of vehicles, readable storage medium, and electronic device | |
| CN105354902B (en) | A kind of security management method and system based on recognition of face | |
| CN104166841B (en) | The quick detection recognition methods of pedestrian or vehicle is specified in a kind of video surveillance network | |
| US11367219B2 (en) | Video analysis apparatus, person retrieval system, and person retrieval method | |
| CN110929066A (en) | Method and device for displaying personnel and package information in association manner and security check machine | |
| CN107424273A (en) | A kind of management method of unmanned supermarket | |
| CN111091098A (en) | Training method and detection method of detection model and related device | |
| CN106650623A (en) | Face detection-based method for verifying personnel and identity document for exit and entry | |
| CN108010242A (en) | A kind of security alarm method, system and storage medium based on video identification | |
| CN113269091A (en) | Personnel trajectory analysis method, equipment and medium for intelligent park | |
| CN106778470A (en) | A kind of face identification method and device | |
| US20230135400A1 (en) | Method and system for facial recognition | |
| CN111191507A (en) | Safety early warning analysis method and system for smart community | |
| Ramesh et al. | An automated vision-based method to detect elephants for mitigation of human-elephant conflicts | |
| CN112949439A (en) | Method and system for monitoring invasion of personnel in key area of oil tank truck | |
| CN111814690A (en) | Target re-identification method and device and computer readable storage medium | |
| CN111833380A (en) | Multi-view image fusion space target tracking system and method | |
| CN113971821A (en) | Driver information determination method and device, terminal device and storage medium | |
| CN113673399A (en) | Method and device for monitoring area, electronic equipment and readable storage medium | |
| CN117292327A (en) | Method, device, equipment and medium for associating targets | |
| CN109214316B (en) | Perimeter protection method and device | |
| Amin et al. | An automatic number plate recognition of Bangladeshi vehicles | |
| CN111814522A (en) | Method and device for processing monitoring image | |
| CN115984830A (en) | Safety belt wearing detection method, device, equipment and storage medium | |
| CN112861711A (en) | Regional intrusion detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |