CN116990874A - Security data association method and device and X-ray security system - Google Patents
Security data association method and device and X-ray security system Download PDFInfo
- Publication number
- CN116990874A CN116990874A CN202310886682.XA CN202310886682A CN116990874A CN 116990874 A CN116990874 A CN 116990874A CN 202310886682 A CN202310886682 A CN 202310886682A CN 116990874 A CN116990874 A CN 116990874A
- Authority
- CN
- China
- Prior art keywords
- ray
- package
- image
- ray image
- image acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000007689 inspection Methods 0.000 claims abstract description 92
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 239000000284 extract Substances 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 230000001815 facial effect Effects 0.000 claims description 8
- 238000010801 machine learning Methods 0.000 claims description 6
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 25
- 238000004891 communication Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 230000001502 supplementing effect Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 239000000470 constituent Substances 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 210000004209 hair Anatomy 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000017525 heat dissipation Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009420 retrofitting Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G01V5/22—
Abstract
The application provides a security inspection data association method and device and an X-ray security inspection system. The security check data association method comprises the following steps: extracting, for each package, an appearance image of the package from an entrance video acquired by an image acquisition device located at an entrance of the X-ray security inspection device; tracking the package through an image tracking technology in a period from the time when the package appears in the field of view of the image acquisition device to the time when the package enters a lead curtain door of the X-ray security inspection device, so as to obtain a motion track of the package; acquiring the moment when the package enters an X-ray security inspection device according to the motion trail; according to the moment when the package enters the X-ray security inspection device, predicting the X-ray image acquisition time in the X-ray video stream acquired by the X-ray security inspection device; and extracting the X-ray image of the package from an X-ray image video stream according to the X-ray image acquisition time, thereby correlating the package appearance image with the X-ray image.
Description
The divisional application is the divisional application of the patent application with the application number of 202110645865.3, the application date of 2021, 6 and 9, and the applicant of Fang Wei, the patent application of the application is entitled "security check data association method and device, and X-ray security check system".
Technical Field
The application relates to the field of security inspection, in particular to a security inspection data association method and device and an X-ray security inspection system.
Background
Currently, in public settings such as airports, subways, stadiums, etc., an X-ray security inspection device is generally used to perform security inspection on baggage carried by an in-out person. Among them, there is also a technology of associating a person with a baggage package through image recognition, thereby determining a person carrying a dangerous article.
For example, patent document 1 discloses a security inspection method based on man-bag association and a device thereof. In patent document 1, in the process of placing a package on a conveyor belt, a video including the passenger and the package is recorded, first associated data of the video, the face image of the passenger and the package appearance image are generated, in order to match the package appearance image and the package X-ray image, a camera is arranged in a security inspection machine, after the package appearance is recorded by the camera, the package is subjected to X-ray scanning, package information is obtained, the package X-ray image is obtained, and then the package appearance image and the package X-ray image are correspondingly generated to second associated data.
Patent document 1: CN111323835a.
Disclosure of Invention
As described above, in patent document 1, the association between the package outline and the X-ray image is established by installing a camera inside the security inspection machine to acquire the package outline and performing X-ray scanning on the package. However, installing the camera inside the security inspection machine requires retrofitting an existing security inspection machine, which increases the cost of the security inspection machine. In addition, when the camera is installed inside, high-power light supplementing equipment is required to be added inside, and the high-power light supplementing equipment increases the power consumption of the security inspection machine and generates larger heat, so that the heat dissipation difficulty of the security inspection machine is increased.
The application can provide a security check data association method and device for improving mismatching and reducing equipment cost, and an X-ray security check system.
One mode of the application provides a security check data association method, which comprises the following steps: extracting, for each package, an appearance image of the package from an entrance video acquired by an image acquisition device located at an entrance of the X-ray security inspection device; tracking the package through an image tracking technology in a period from the time when the package appears in the field of view of the image acquisition device to the time when the package enters a lead curtain door of the X-ray security inspection device, so as to obtain a motion track of the package; according to the motion trail, the moment when the package enters the X-ray security inspection device is obtained; according to the moment when the package enters the X-ray security inspection device, predicting the X-ray image acquisition time in the X-ray video stream acquired by the X-ray security inspection device; and extracting the X-ray image of the package from an X-ray image video stream according to the X-ray image acquisition time, thereby correlating the package appearance image with the X-ray image.
In the security inspection data association method, the X-ray image acquisition time and the X-ray image acquisition position are predicted according to the moment when the package enters the X-ray security inspection device in the motion track, wherein the X-ray image acquisition position represents the position of the X-ray image of the package in an X-ray video stream, and the X-ray image of the package is extracted from the X-ray image video stream according to the X-ray image acquisition time and the X-ray image acquisition position.
In the above security inspection data association method, predicting the X-ray image acquisition time and the X-ray image acquisition position according to the time when the package enters the X-ray security inspection device in the motion track includes:
according to the position and time of the corresponding package entering the lead curtain door of the X-ray security inspection device in the motion trail and the motion speed of the conveyor belt, the X-ray image acquisition time and the X-ray image acquisition position of the X-ray image of the package are calculated, and according to the X-ray image acquisition time and the X-ray image acquisition position, the X-ray image of the package is extracted from an X-ray image video stream, wherein the X-ray image extraction comprises: and dividing the X-ray video stream according to the X-ray image acquisition time, the X-ray image acquisition position and the appearance size of the package to obtain the X-ray image of the package.
In the above security inspection data association method, dividing the X-ray video stream according to the X-ray image acquisition time, the X-ray image acquisition position, and the appearance size of the package, the obtaining the X-ray image of the package includes: and extracting a corresponding X-ray video frame according to the occurrence time of the X-ray image in an X-ray video stream, and dividing the extracted X-ray video frame according to the occurrence position of the X-ray image in the X-ray video frame to obtain the X-ray image of the package, wherein the dividing size is determined by the actual size of the package and the display proportion of the X-ray security inspection device.
In the above security inspection data association method, the time length of the period is set according to the setting position of the image acquisition device, the shooting range and the time length requirement of motion track tracking.
The security inspection data association method further comprises the following steps:
extracting a human body image from an entrance video acquired at an entrance of an X-ray security inspection device, correlating the human body image of the same package with the package appearance image, automatically extracting the characteristics from the human body image through a machine learning method, and identifying whether the human body image is the same person according to the extracted characteristics, wherein the human body image contains various characteristics representing the whole and/or part of human body information.
In the above-mentioned security inspection data association method, the human body image contains both the whole feature and the local feature, and the local feature cannot contain only the facial feature.
Another aspect of the present application provides a security check data association apparatus, including: the appearance image acquisition unit is used for extracting appearance images of the packages from the entrance video acquired by the image acquisition device at the entrance of the X-ray security inspection device for each package; the tracking unit is used for tracking the package through an image tracking technology from the time when the package appears in the field of view of the image acquisition device to the time when the package enters the lead curtain door of the X-ray security inspection device, so as to obtain the motion trail of the package; the prediction unit is used for obtaining the moment when the package enters the X-ray security inspection device according to the motion trail and predicting the X-ray image acquisition time in the X-ray video stream acquired by the X-ray security inspection device according to the moment when the package enters the X-ray security inspection device; and an X-ray image extraction unit for extracting X-ray images of the package from an X-ray image video stream according to the X-ray image acquisition time, thereby associating the package appearance image with the X-ray image.
In the above security inspection data association device, the prediction unit predicts the X-ray image acquisition time and the X-ray image acquisition position according to the time when the package enters the X-ray security inspection device in the motion track, where the X-ray image acquisition position indicates a position of the X-ray image of the package in the X-ray video stream, and the X-ray image extraction unit extracts the X-ray image of the package from the X-ray image video stream according to the X-ray image acquisition time and the X-ray image acquisition position.
In the above security inspection data association device, the prediction unit calculates the X-ray image acquisition time and the X-ray image acquisition position of the X-ray image of the package according to the position and the time of the corresponding package entering the lead shutter door of the X-ray security inspection device in the motion track and the motion speed of the conveyor belt, and the X-ray image extraction unit divides the X-ray video stream according to the X-ray image acquisition time and the X-ray image acquisition position and the appearance size of the package to obtain the X-ray image of the package.
In the above security inspection data association device, the X-ray image extracting unit extracts a corresponding X-ray video frame according to a time when the X-ray image appears in an X-ray video stream, and segments the extracted X-ray video frame according to a position where the X-ray image appears in the X-ray video frame, so as to obtain the X-ray image of the package, wherein a size of the segmentation is determined by an actual size of the package and a display ratio of the X-ray security inspection device.
In the security inspection data association device, the time length of the period is set according to the setting position of the image acquisition device, the shooting range and the time length requirement of motion track tracking.
The above-mentioned security inspection data association device further includes: the human package association unit is used for extracting human body images from entrance videos acquired at the entrance of the X-ray security inspection device and associating the human body images of the same package with the package appearance images; and a human body recognition unit that automatically extracts the features from the human body image by a machine learning method, and recognizes whether the human body is the same person based on the extracted features, wherein the human body image contains a plurality of features representing the whole and/or part of human body information.
In the above-mentioned security inspection data association device, the human body image includes both the whole feature and the local feature, and the local feature cannot include only the facial feature.
Another aspect of the present application provides an X-ray security inspection system, comprising: the security inspection data association device; the image acquisition device is arranged at the inlet of the X-ray security inspection device and acquires videos at the inlet; and an X-ray security inspection device for generating an X-ray video stream by X-rays.
In the application, the association of the appearance image of the package and the X-ray image is realized by the image tracking technology, so that a camera, a light supplementing device and the like do not need to be arranged in the X-ray security inspection device, thereby simplifying the equipment structure and reducing the equipment cost.
Drawings
FIG. 1 is a schematic diagram showing the construction of an X-ray security system;
fig. 2 is a functional block diagram showing a security data-relating apparatus according to the first embodiment;
fig. 3 is a flowchart showing a security check data association method according to the first embodiment;
fig. 4 is a flowchart showing a specific example of the step S303;
fig. 5 is a functional block diagram showing a security data-relating apparatus according to a second embodiment;
fig. 6 is a flowchart showing a security check data association method according to the second embodiment.
Detailed Description
Exemplary embodiments or examples of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments or examples set forth herein. Rather, these embodiments or examples are provided so that this disclosure will be understood more clearly.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the implementations or embodiments of the application described herein may be implemented in other sequences than those illustrated or otherwise described. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed. The same or similar reference numerals herein denote constituent elements having the same or similar functions.
< first embodiment >
As shown in fig. 1, the X-ray security inspection system 1 includes: conveyor belt 11, X-ray machine 12, image acquisition device 13.
When a passer-by passes through the security inspection channel 20 beside the X-ray security inspection system 1 in the arrow direction in fig. 1, a carry-on baggage or a package (hereinafter, simply referred to as "package") is put on the conveyor belt 11, and at this time, the image pickup device 13 provided at the entrance of the X-ray machine 12 picks up the entrance video, and the conveyor belt 11 moves in the horizontal direction (from left to right in fig. 1) and passes through the inside of the X-ray machine 12, and the X-ray machine 12 scans the package entering the inside to generate an X-ray image.
Specifically, each constituent part of the X-ray security inspection system 1 is as follows.
The conveyor belt 11 may be configured to convey packages placed thereon in a horizontal direction.
The X-ray machine 12 may X-ray the package within its scanning range to generate an X-ray image. Since the X-ray machine 12 scans packages that are continuously passed along with the movement of the conveyor belt 11, the X-ray images generated may be a stream of X-ray video, i.e. a sequence of X-ray video frames.
The conveyor belt 11 and the X-ray machine 12 may constitute an X-ray security device.
The image acquisition device 13 is mounted at the entrance of the X-ray machine. And carrying out image acquisition on the people and the packages in the image acquisition area. In other words, a portal video containing people and parcels passing through the image acquisition area is captured.
In an exemplary embodiment, the image capturing device 13 may be a camera, more specifically, a still picture capturing camera, or a video capturing camera. In other words, the image may be a still image or a moving image. Preferably, the image acquisition device 13 acquires still images which may be video or continuously shot. In this case, the image pickup device 13 may dynamically photograph the package placed on the conveyor belt 11 and the human body on which the package is placed, and thus, the image picked up at this time may be a sequence of visible light video frames.
The conveyor belt 11, the X-ray machine 12, and the image pickup device 13 may be conventional ones.
The X-ray security system 1 may further comprise: and a control device. The control device may be connected to at least one of the conveyor belt 11, the X-ray machine 12, and the image pickup device 13, respectively, and may control at least one of the movement of the conveyor belt 11, the scanning of the X-ray machine 12, the photographing of the image pickup device 13, and the like.
The X-ray security system 1 may further comprise: and a security data association device which may be part of the control device or may be provided independently of the control device. The security check data association device is connected with at least the X-ray machine 12 and the image acquisition device 13.
The security data association means may be implemented in hardware or in software modules running on one or more processors or in a combination thereof. For example, the security check data association device may be implemented by any suitable electronic device, such as a desktop computer, a tablet computer, a smart phone, a server, etc. provided with a processor, in a combination of hardware and software.
The processor of the security data association device may execute each step in the security data association method described later or a program corresponding to each step.
The security data association device may further include a memory (not shown) and a communication module (not shown), etc.
The memory of the security data association device may store the steps of executing the security data association method described later, data related to the security data association, and the like. The memory may be, for example, ROM (Read Only Memory image, read only memory), RAM (Random Access Memory ), or the like. The memory has memory space for program code for performing any of the following security data association methods. These program codes, when read and executed by the processor, perform the security check data association method described below. The program code can be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, compact Disk (CD), memory card or floppy disk. Such computer program products are typically portable or fixed storage units. Program code for performing any of the steps in the following methods may also be downloaded over a network. The program code may be compressed, for example, in a suitable form.
The communication module in the security data association device may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the security data association device and the external electronic device and performing communication via the established communication channel. For example, the communication module receives video at the entrance from the image pickup device 13 via a network, X-ray images from the X-ray machine 12 via a network, and the like.
In addition, the security data association device may further include an output part such as a display, a microphone, a speaker, etc. for displaying the security data association result, or issuing a warning, etc.
Hereinafter, specific embodiments of a security data association apparatus and method will be described with reference to the accompanying drawings.
< first embodiment >
Fig. 2 is a functional block diagram schematically showing a security data correlation device according to the first embodiment. These functional modules may be implemented by a processor of the security data association device executing a program stored in a memory of the security data association device.
As shown in fig. 2, the security data association device 14 includes: the human package association unit is used for extracting a human body image and a package appearance image from an entrance video acquired at an entrance of the X-ray security inspection device and associating the human body image and the package appearance image corresponding to the same package; the package image association unit is used for tracking each package in the video at the entrance to obtain a motion trail of each package, and associating the package appearance image and the X-ray image of the same package according to the motion trail; and a person package image association unit for associating the human body image, the package appearance image and the X-ray image corresponding to the same package through the package appearance image.
A security data association method performed by the security data association device in the first embodiment will be described below with reference to fig. 3.
In step S301, the human package association unit 141 extracts a human body image and a package appearance image from an entrance video acquired at an entrance of the X-ray security inspection apparatus, and associates the human body image and the package appearance image corresponding to the same package.
Here, the human body image and the package appearance image corresponding to the person contained in the human body image can be correspondingly extracted from the video at the entrance, so that the association between the human body image and the package appearance image is realized. Or after extracting the human body image and the parcel appearance image from the video at the entrance, searching the corresponding relation between the human body image and the parcel appearance image, and correlating the human body image and the parcel appearance image.
Here, the package association data may be further generated from the human body image and the corresponding package appearance image, the package association data representing a correspondence relationship between the human body image and the package appearance image.
In this embodiment, the human body image and the corresponding parcel appearance image are extracted and associated from the entrance video acquired by the image acquisition device 13.
Here, the human body image contains various features representing the whole and/or part of the human body information. That is, the human body image may be an image including the entire feature of the human body, an image including the partial feature of the human body, or an image including both the entire feature and the partial feature.
Wherein it is preferred to include both global and local features. In such a case, the overall feature and the local feature can be combined, mutual authentication can be performed, and accuracy of personnel identification can be further improved. In addition, even when a part of the human body image is blocked, the same person can be identified by other features, and the reliability of person identification can be improved.
The overall feature here is a feature that can distinguish a person from other persons as a whole, for example, a person's outline, height, weight, clothing style, etc.
Local features are features that can locally distinguish a person from other people, such as eyebrows, eyeglasses, mouth, hair, hats, hoods, and the like.
Such various features may be automatically extracted from the human body image by a machine learning method. In the present application, the human body image may be a whole-body image, a half-body image, or a whole-body or half-body image partially blocked, as long as it includes various features of the whole body and/or part of the human body information.
Here, the security data association device 14 may further include a human body recognition unit. The human body recognition unit may automatically extract the whole and partial features of the human body from the human body image by a machine learning method, and recognize whether the same person is based on the extracted features. The person-pack associating unit 141 may extract a group of human body images and package appearance images for the same person according to the recognition result of the human body recognizing unit, and associate them.
Here, when judging whether or not the same person is present, the comparison may be performed with only the human body images of the n persons before and after, and the comparison may not be performed with all the human body images stored in the memory. In other words, the human body recognition unit compares only human body images within a predetermined delay time range, and recognizes whether or not the human body images are the same person. The predetermined delay time may be preset according to the security inspection process time of the X-ray machine 12. This is because, as in the present application, generally, the time for a person to complete a security inspection process by using the X-ray machine 12 is within a certain time range, and the time delay range of the human body image is set according to the time of such security inspection process, and only the matching of the human body image is performed within the time delay range, so that the mismatching can be effectively reduced.
In addition, in the present application, it is preferable that the local feature cannot include only the facial feature. In other words, the human body image may be a larger-range image including the face, or may be an image of a part other than the face, unlike the face image. More preferably, the human body image is a whole image or an upper body image. Because whether the person is identified through the human body image, the face of the person is shielded by objects such as colored glasses (e.g. sunglasses), scarves or masks, so that the person can be identified through other human body characteristics under the condition that the person cannot be identified by using the facial characteristics, the person identification requirement of a security inspection system can be met, and the mismatching rate can be reduced.
In addition, since the image acquisition device 13 recognizes whether the person is the same person or not by the human body image, the person can be recognized according to the whole or partial features of the person, and therefore, unlike the case of recognizing the same person only by the face recognition, the person is not required to be individually captured, and when different passers-by passes, the angle of the image acquisition device 13 does not need to be continuously adjusted in order to recognize the face, so that the cradle head device is not required, and the image acquisition device 13 which is fixedly installed is favorable to continuously capture the image without the cradle head device, and the condition that the image shake and the like do not occur in the video is continuously captured.
However, in the related art such as patent document 1, first, the same person is identified by the face image of the passenger, and the association relationship between the person and the package is found. However, in the security scene, the face is easily blocked by an object such as a hat, a pair of colored glasses (e.g., a sunglasses), a scarf, or a mask, and thus, in the case of only face recognition as in patent document 1, there is a possibility that serious mismatching may occur. Secondly, under the condition that the same person is identified only through face recognition, the requirement on shooting facial images is high, and the angle of the camera needs to be continuously adjusted so as to record clear facial images. Therefore, the pan-tilt equipment for adjusting the angle of the camera becomes a necessary component, so that the cost of the security inspection device is increased, in addition, the camera angle is continuously adjusted in the human body movement process, the movement is easy to be blurred, and clear face images and parcel images are difficult to collect.
As described above, in the present application, the same person is identified by using the human body image instead of the face image, and thus, it is possible to reduce mismatching due to inaccurate face recognition during occlusion, and it is not necessary to provide an angle adjustment device such as a cradle head for acquiring the human body image, so that the mismatching rate can be further reduced, and the device cost can be further reduced. In step S302, the tracking unit 142 tracks the packages in the video at the entrance to obtain the motion trail of each package.
In particular, the trajectory of the package may be acquired during a period from when the package is present in the field of view of the image acquisition device 13 to when it enters the lead door of the X-ray machine 12. Here, the length of time during this period may be appropriately set according to the setting position of the image pickup device 13, the shooting range, the time length requirement of the movement locus tracking, and the like.
Here, the movement locus may be a continuous movement locus for a time from when the package appears in the field of view of the image pickup device 13 to when the package enters the lead shutter door of the X-ray machine 12, or may be a plurality of intermittent movement loci. Preferably, a motion profile is used that includes the point in time at which the package enters the lead shutter door of the X-ray machine 12.
In the case where the period does not include the time point at which the package enters the lead shutter door of the X-ray machine 12 but ends before it, the next movement locus may be predicted from the movement locus during the period, thereby obtaining the movement locus of the time point or the like at which the package enters the lead shutter door or the like of the X-ray machine 12.
The tracking technique may use any technique already known.
In step S303, the package map associating unit 143 associates the package appearance image and the X-ray image of the same package according to the motion trajectory. Here, the package appearance image of the package and the X-ray image may be associated according to a motion track of the package to generate package map association data, where the package map association data represents a correspondence between the package appearance image and the X-ray image.
A specific example of step S303 will be described below with reference to fig. 4.
In step S3031, the tracking section predicts an X-ray image acquisition time and an X-ray image acquisition position at which the X-ray image can be acquired, based on the motion trajectory acquired in step S302.
Specifically, the X-ray image acquisition time and the X-ray image acquisition position are predicted according to the moment when the package enters the X-ray security inspection device in the motion trail. Wherein the X-ray image acquisition time indicates a time when the X-ray image of the package appears in the X-ray video stream and the X-ray image acquisition location indicates a location where the X-ray image of the package appears in the X-ray video stream.
More specifically, the X-ray image acquisition time and the X-ray image acquisition position of the X-ray image of the package can be calculated from the position of the lead shutter door of the X-ray machine 12 and the time of the package entering the lead shutter door of the X-ray machine 12 in the movement locus, and the movement speed of the conveyor belt 11. In this case, the time of day of the X-ray security device is the time at which the package entered the lead door of the X-ray machine 12. The X-ray image acquisition time and the X-ray image acquisition position may be predicted from the time when the package in the movement locus enters the X-ray irradiation region in the X-ray machine 12, the time when the package reaches a specific position in front of the lead shutter, or the like.
In step S3032, the association unit extracts the X-ray image of the package from the X-ray image video stream according to the X-ray image acquisition time and the X-ray image acquisition position of the X-ray image and the actual size of the package.
Specifically, the corresponding X-ray video frame may be extracted according to the time when the X-ray image appears in the X-ray video stream, and then the extracted X-ray video frame may be segmented according to the position where the X-ray image appears in the X-ray video frame, so as to obtain the X-ray image of the package, where the size of the segmentation is determined by the actual size of the package and the display ratio of the X-ray security inspection device.
In step S3033, the association unit associates the package appearance image of the package corresponding to the movement trace with the X-ray image.
In step S304, the person-package-map associating unit 144 associates the person image, the package appearance image, and the X-ray image corresponding to the same package via the package appearance image. Here, the package map association data representing the correspondence relationship among the human body image, the package appearance image, and the X-ray image may be generated by associating the human body image, the package appearance image, and the X-ray image corresponding to the same package.
In step S301, the correspondence between the human body image and the package appearance image is obtained, and in step S303, the correspondence between the package appearance image and the X-ray image is obtained, so that the three images of the human body image, the package appearance image, and the X-ray image corresponding to the same package can be associated via the package appearance image, in other words, with the package appearance image as an intermediary.
As described above, the X-ray image corresponding to the package appearance image is obtained by the tracking technique, and the package appearance image and the X-ray image can be associated without providing an imaging device, a light supplementing device, or the like inside the X-ray apparatus 12, or without changing the structure of the existing X-ray apparatus 12, thereby simplifying the device structure and reducing the device cost.
< second embodiment >
Fig. 5 is a functional block diagram schematically showing a security data correlation device according to a second embodiment.
As shown in fig. 5, the security data association device 14' may further include: a graph judging unit 145, for obtaining a graph judging result according to the X-ray image, wherein the graph judging result indicates whether the package contains contraband; and a graph judgment result association unit 146 that associates the human body image, the package appearance image, the X-ray image, and the graph judgment result corresponding to the same package.
Here, the human body image, the package appearance image, the X-ray image, and the image judgment result corresponding to the same package may be correlated to generate package image judgment correlation data indicating correspondence between the human body image, the package appearance image, the X-ray image, and the image judgment result.
In addition, if the package contains contraband according to the graph judging result, the graph judging result can also contain contraband information, the contraband information comprises at least one of the position of the contraband and the type, the number and the like of the contraband, and the contraband information can be structured data generated by structuring the position of the contraband and the type, the number and the like of the contraband. The graph judgment result can be obtained through manual graph judgment or intelligent recognition. When the images are judged manually, the contraband information can also contain the ID of image judging personnel and the like, and when the images are identified intelligently, the contraband information can also contain the adopted identification method and the like.
The contraband information and the personal package image association data can be associated to generate the personal package image judgment association data representing the corresponding relationship among the human body image, the package appearance image, the X-ray image and the judgment result.
In addition, the generated personnel package graph judgment association data can be displayed on a real-time display interface of a security check site, and when the contraband in the package is judged, the system can send out audible and visual alarm. In addition, the generated package graph judgment association data can be uploaded to a database for storage and can be used for subsequent inquiry and tracing.
The human body image, the package appearance image, the X-ray image and the image judging result corresponding to the same package are correlated, so that the human body image, the package appearance image and the X-ray image corresponding to the package can be easily checked for packages containing contraband, people carrying the contraband can be rapidly identified through the human body image, packages containing the contraband can be rapidly locked through the package appearance image, and the contraband can be manually confirmed through the X-ray image.
Other structures of the security data association apparatus of the second embodiment may be the same as those of the first embodiment, except for this.
Fig. 6 is a flowchart showing an overall flow of a security check data association method according to the second embodiment.
In fig. 6, in addition to steps S301 to S304, the steps include: s305 step and S306 step.
In step S305, the image determining unit 145 obtains a result of determining an image from the X-ray image, where the result of determining an image indicates whether the package contains contraband information. In addition, the graph judgment result can also contain information such as contraband types, positions and the like.
In step S306, the image judgment result association unit 146 associates the human body image, the package appearance image, the X-ray image, and the image judgment result corresponding to the same package, and generates the human package image judgment association data.
In fig. 6, step S301 may be performed by dividing step S3011 of extracting a human body image and a package appearance image from an entrance video acquired at an entrance of the X-ray security inspection apparatus, and step S3012 of generating package association data by associating the human body image and the package appearance image based on the extracted human body image and package appearance image.
As shown in fig. 6, the process of generating the person-pack association data in step S301 and step S303 may be performed in parallel. After the step S3011 is completed, the extracted package appearance image may be provided to the subsequent steps S302 and S303, without waiting for the completion of the step S3012. Thus, the processing speed can be increased, and the efficiency can be improved.
In addition, the security check data association device may further include: and an output unit outputting the associated data. The association data may be the above-mentioned personal packet map association data or personal packet map judgment association data, or may be the number of personal packet associations or packet map association data. As shown in fig. 6, step S307 may also be included. In step S307, the output unit outputs the associated data.
In addition, the X-ray security inspection system may further include: the exit image acquisition device is arranged at the exit of the X-ray machine 12 and acquires images containing the human body and the package. At this time, the security check data association device may further include an exit comparison unit. The exit alignment unit includes: an outlet image extraction part for extracting a human body image and a package appearance image at the outlet from the outlet acquired image; the exit person-package data association part is used for associating the extracted human body image with the corresponding package appearance image to obtain exit person-package association data; and a comparison unit for comparing the entrance-person-package related data with the exit-person-package related data. At this time, the output association data may also include a comparison result of the comparison unit at the exit, and the security check data association device may also issue an alarm when the person packet association data at the entrance and the person packet association data at the exit are inconsistent. Therefore, the situation that the passers-by takes the package by mistake or fails to take the package at the exit can be detected.
In addition, although the case where the human body recognition unit is included only in the man-bag association unit 141 is described in the first embodiment, when the security data association device includes the exit comparison unit, the exit comparison unit includes the human body recognition unit and the package recognition unit. Specifically, the above-mentioned comparing section of the exit comparing unit may be provided with a human body recognizing unit and a parcel recognizing unit. The human body recognition unit compares whether the human body image at the exit and the human body image at the entrance represent the same person according to the recognition result of the human body recognition unit. And similarly, the parcel identifying unit compares the parcel appearance image at the outlet with the parcel appearance image at the inlet, and judges whether the parcels are the same parcel or not according to the parcel appearance image. The exit comparison unit can judge whether the packages taken by the person are the same or not through the package identification unit on the basis that the person is judged to be the same through the human body identification unit.
In addition, the output unit may further include an alarm unit that issues an alarm when the result of the map judgment indicates that the package contains contraband; or when the comparison result of the comparison unit at the outlet is abnormal, an alarm is sent out. Therefore, when the packages contain contraband, or when personnel mishold or miss the packages, an alarm is given, so that the situation that the contraband is wrongly released or the passing personnel mishold or miss the packages is avoided.
Although the embodiments and specific examples of the present application have been described above with reference to the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the application, and such modifications and variations fall within the scope defined by the claims.
Claims (15)
1. A security data association method, comprising:
extracting, for each package, an appearance image of the package from an entrance video acquired by an image acquisition device located at an entrance of the X-ray security inspection device;
tracking the package through an image tracking technology in a period from the time when the package appears in the field of view of the image acquisition device to the time when the package enters a lead curtain door of the X-ray security inspection device, so as to obtain a motion track of the package;
acquiring the moment when the package enters an X-ray security inspection device according to the motion trail;
according to the moment when the package enters the X-ray security inspection device, predicting the X-ray image acquisition time in the X-ray video stream acquired by the X-ray security inspection device; and
and extracting the X-ray image of the package from an X-ray image video stream according to the X-ray image acquisition time, thereby correlating the appearance image of the package with the X-ray image.
2. The security data association method of claim 1, wherein,
predicting the X-ray image acquisition time and the X-ray image acquisition position according to the moment when the package enters the X-ray security inspection device in the motion trail, wherein the X-ray image acquisition position represents the position of the X-ray image of the package in an X-ray video stream,
and extracting the X-ray image of the package from an X-ray image video stream according to the X-ray image acquisition time and the X-ray image acquisition position.
3. The security data association method of claim 2, wherein,
according to the moment when the package enters the X-ray security inspection device in the motion trail, the X-ray image acquisition time and the X-ray image acquisition position are predicted, wherein the method comprises the following steps:
calculating the X-ray image acquisition time and the X-ray image acquisition position of an X-ray image of the package according to the position and time of the corresponding package entering the lead curtain door of the X-ray security inspection device in the motion trail and the motion speed of the conveyor belt,
extracting the X-ray image of the package from the X-ray image video stream according to the X-ray image acquisition time and the X-ray image acquisition location comprises:
and dividing the X-ray video stream according to the X-ray image acquisition time, the X-ray image acquisition position and the appearance size of the package to obtain the X-ray image of the package.
4. A security data association method as claimed in claim 3 wherein,
dividing the X-ray video stream according to the X-ray image acquisition time, the X-ray image acquisition position and the appearance size of the package, and obtaining the X-ray image of the package includes:
extracting corresponding X-ray video frames based on the time at which the X-ray image appears in the X-ray video stream,
dividing the extracted X-ray video frame according to the position of the X-ray image in the X-ray video frame to obtain the X-ray image of the package,
the size of the division is determined by the actual size of the package and the display proportion of the X-ray security inspection device.
5. The security data association method of claim 1, wherein,
the time length of the period is set according to the setting position of the image acquisition device, the shooting range and the time length requirement of motion trail tracking.
6. The security data association method of claim 1, further comprising:
extracting a human body image from an entrance video acquired at an entrance of an X-ray security inspection device, correlating the human body image of the same package with the package appearance image,
automatically extracting the features from the human body image by a machine learning method, identifying whether the same person is based on the extracted features,
wherein the human body image contains a plurality of features representing the whole and/or part of the human body information.
7. The security data association method of claim 6, wherein,
the human body image contains both global features and local features, which cannot contain only facial features.
8. A security data association device, comprising:
the appearance image acquisition unit is used for extracting appearance images of the packages from the entrance video acquired by the image acquisition device at the entrance of the X-ray security inspection device for each package;
the tracking unit is used for tracking the package through an image tracking technology from the time when the package appears in the field of view of the image acquisition device to the time when the package enters the lead curtain door of the X-ray security inspection device, so as to obtain the motion trail of the package;
the prediction unit is used for obtaining the moment when the package enters the X-ray security inspection device according to the motion trail and predicting the X-ray image acquisition time in the X-ray video stream acquired by the X-ray security inspection device according to the moment when the package enters the X-ray security inspection device; and
and the X-ray image extraction unit is used for extracting the X-ray image of the package from an X-ray image video stream according to the X-ray image acquisition time, so that the appearance image of the package and the X-ray image are associated.
9. The security data association device of claim 8, wherein,
the prediction unit predicts the X-ray image acquisition time and the X-ray image acquisition position according to the moment when the package enters the X-ray security inspection device in the motion trail, wherein the X-ray image acquisition position represents the position of the X-ray image of the package in an X-ray video stream,
and the X-ray image extraction unit extracts the X-ray image of the package from an X-ray image video stream according to the X-ray image acquisition time and the X-ray image acquisition position.
10. The security data association device of claim 9, wherein,
the prediction unit calculates the X-ray image acquisition time and the X-ray image acquisition position of the X-ray image of the package according to the position and time of the lead curtain door of the X-ray security inspection device, which correspond to the package in the motion trail, and the motion speed of the conveyor belt,
the X-ray image extraction unit divides the X-ray video stream according to the X-ray image acquisition time, the X-ray image acquisition position and the appearance size of the package to obtain the X-ray image of the package.
11. The security data association device of claim 10, wherein,
the X-ray image extraction unit extracts a corresponding X-ray video frame according to the occurrence time of the X-ray image in an X-ray video stream, and segments the extracted X-ray video frame according to the occurrence position of the X-ray image in the X-ray video frame to obtain the X-ray image of the package, wherein the segmentation size is determined by the actual size of the package and the display proportion of the X-ray security inspection device.
12. The security data association device of claim 8, wherein,
the time length of the period is set according to the setting position of the image acquisition device, the shooting range and the time length requirement of motion trail tracking.
13. The security data association device of claim 1, further comprising:
the human package association unit is used for extracting human body images from entrance videos acquired at the entrance of the X-ray security inspection device and associating the human body images of the same package with the package appearance images; and
a human body recognition unit automatically extracting the features from the human body image by a machine learning method, recognizing whether the same person is based on the extracted features,
wherein the human body image contains a plurality of features representing the whole and/or part of the human body information.
14. The security data association device of claim 13 wherein,
the human body image contains both global features and local features, which cannot contain only facial features.
15. An X-ray security system comprising:
a security data association device as claimed in any one of claims 8 to 14;
the image acquisition device is arranged at the entrance of the X-ray security inspection device and acquires videos at the entrance; and
the X-ray security inspection device generates the X-ray video stream through X-rays.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310886682.XA CN116990874A (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310886682.XA CN116990874A (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
| CN202110645865.3A CN113759431B (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110645865.3A Division CN113759431B (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116990874A true CN116990874A (en) | 2023-11-03 |
Family
ID=78787435
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110645865.3A Active CN113759431B (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
| CN202310886682.XA Pending CN116990874A (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110645865.3A Active CN113759431B (en) | 2021-06-09 | 2021-06-09 | Security data association method and device and X-ray security system |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN113759431B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113267828A (en) * | 2021-06-09 | 2021-08-17 | 杭州海康威视数字技术股份有限公司 | Information association method and device, security check equipment and storage medium |
| CN115494556B (en) * | 2022-08-18 | 2023-09-12 | 成都智元汇信息技术股份有限公司 | Packet association method based on paragraph fuzzy matching |
| CN115422391B (en) * | 2022-08-18 | 2023-05-26 | 成都智元汇信息技术股份有限公司 | Person-package association method and device based on graph searching |
| CN115423996B (en) * | 2022-08-18 | 2023-09-22 | 成都智元汇信息技术股份有限公司 | Method, electronic equipment and medium for extracting parcel pictures based on parcel movement |
| CN115496889B (en) * | 2022-08-18 | 2023-09-22 | 成都智元汇信息技术股份有限公司 | Method, equipment and medium for extracting package picture in X-ray picture based on RGB image |
| CN115438208A (en) * | 2022-08-23 | 2022-12-06 | 成都智元汇信息技术股份有限公司 | Method and device for improving package association accuracy |
| CN115469375B (en) * | 2022-11-14 | 2023-02-14 | 成都智元汇信息技术股份有限公司 | System for associating visible light image and X-ray image of person bag |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6554189B1 (en) * | 1996-10-07 | 2003-04-29 | Metrologic Instruments, Inc. | Automated system and method for identifying and measuring packages transported through a laser scanning tunnel |
| US6382515B1 (en) * | 1995-12-18 | 2002-05-07 | Metrologic Instruments, Inc. | Automated system and method for identifying and measuring packages transported through a laser scanning tunnel |
| JP5864404B2 (en) * | 2012-12-07 | 2016-02-17 | 株式会社イシダ | X-ray inspection equipment |
| CN106934326B (en) * | 2015-12-29 | 2020-07-07 | 同方威视技术股份有限公司 | Method, system and device for security check |
| CN106845385A (en) * | 2017-01-17 | 2017-06-13 | 腾讯科技（上海）有限公司 | The method and apparatus of video frequency object tracking |
| CN206863249U (en) * | 2017-07-06 | 2018-01-09 | 上海英曼尼安全装备有限公司 | A kind of rays safety detection apparatus for face recognition |
| US10546197B2 (en) * | 2017-09-26 | 2020-01-28 | Ambient AI, Inc. | Systems and methods for intelligent and interpretive analysis of video image data using machine learning |
| CN111612020B (en) * | 2019-02-22 | 2024-04-26 | 杭州海康威视数字技术股份有限公司 | Positioning method for abnormal object to be detected, security inspection analysis equipment and security inspection analysis system |
| IT201900019454A1 (en) * | 2019-10-21 | 2021-04-21 | Microtec Srl | METHOD AND EQUIPMENT FOR PERFORMING A TOMOGRAPHIC EXAMINATION OF AN OBJECT |
| CN110929066A (en) * | 2019-11-18 | 2020-03-27 | 浙江大华技术股份有限公司 | Method and device for displaying personnel and package information in association manner and security check machine |
| CN111323835B (en) * | 2020-03-20 | 2022-09-16 | 安徽启新明智科技有限公司 | Security inspection method and device based on personal bag association |
| CN112364903A (en) * | 2020-10-30 | 2021-02-12 | 盛视科技股份有限公司 | X-ray machine-based article analysis and multi-dimensional image association method and system |
-
2021
- 2021-06-09 CN CN202110645865.3A patent/CN113759431B/en active Active
- 2021-06-09 CN CN202310886682.XA patent/CN116990874A/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN113759431B (en) | 2023-08-11 |
| CN113759431A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113759431B (en) | Security data association method and device and X-ray security system | |
| US11704936B2 (en) | Object tracking and best shot detection system | |
| US11157778B2 (en) | Image analysis system, image analysis method, and storage medium | |
| CN108256459A (en) | Library algorithm is built in detector gate recognition of face and face based on multiple-camera fusion automatically | |
| CN104361327B (en) | A kind of pedestrian detection method and system | |
| US8866931B2 (en) | Apparatus and method for image recognition of facial areas in photographic images from a digital camera | |
| KR101215948B1 (en) | Image information masking method of monitoring system based on face recognition and body information | |
| US9124778B1 (en) | Apparatuses and methods for disparity-based tracking and analysis of objects in a region of interest | |
| US8600121B2 (en) | Face recognition system and method | |
| CN109272347A (en) | A kind of statistical analysis technique and system of shops's volume of the flow of passengers | |
| US20190180111A1 (en) | Image summarization system and method | |
| CN110969118B (en) | Track monitoring system and method | |
| CN110633648B (en) | Face recognition method and system in natural walking state | |
| CN111325133B (en) | Image processing system based on artificial intelligent recognition | |
| CN109840962A (en) | Recognition of face movable attendance checking system and method | |
| US20180046866A1 (en) | Method of Detecting a Moving Object by Reconstructive Image Processing | |
| CN108875490A (en) | The method, apparatus and computer storage medium of personnel's flow analysis | |
| KR20170006356A (en) | Method for customer analysis based on two-dimension video and apparatus for the same | |
| JP2017211731A (en) | Head-count counting system, head-count counting method, and head-count counting result browsing method | |
| Islam et al. | Correlating belongings with passengers in a simulated airport security checkpoint | |
| CN110930432A (en) | Video analysis method, device and system | |
| CN114677608A (en) | Identity feature generation method, device and storage medium | |
| JP2005140754A (en) | Method of detecting person, monitoring system, and computer program | |
| KR101926510B1 (en) | Wide area surveillance system based on facial recognition using wide angle camera | |
| CN208636897U (en) | A kind of safety check apparatus based on face identification functions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |