JP2019110474A - Suspicious person detection device - Google Patents

Abstract

To provide a suspicious person detection device capable of detecting a suspicious person in consideration of privacy as well as reducing the amount of processing of verification processing in a region where many unspecified persons gather.SOLUTION: A person tracking unit 32 detects one or more persons from each of a plurality of images included in a time-series image to set a person region, and performs tracking processing of each person region between the plurality of images included in the time-series image. A face detection unit 34 analyzes each of the person regions in each of the images to detect a face image of a person included in the person region, and determines whether the type of the face image is a hidden face image or a bare face image, or a determinable image. A face image comparison unit 36 performs verification processing of checking the feature amount of the bare face image of only a face state change person who is a person in which both the hidden face image and the bare face image are detected from the plurality of images included in the time-series image 40 against the face feature amount of the suspicious person stored in a suspicious person information DB26.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a suspicious person detection device that detects a suspicious person by collating a face image.

  Recently, one of the targets to be monitored in an area where an unspecified number of people gather such as an airport, a station, and an event hall, there is a suspicious person such as an arranger.

  Conventionally, detection of a suspicious person has been performed by collating face images of all persons present in an area where surveillance is intensified with face feature information of a known suspicious person.

  For example, in the image monitoring system described in Patent Document 1, area information including the setting of a detailed monitoring area which needs to be particularly monitored is stored, and a face image of a wanted offender (that is, a suspicious person) The individual feature extraction unit extracts the face image of the person present in the detail monitoring area, and the determination unit extracts the extracted face image and the face image stored as the abnormality determination data. The person who matched and matched was judged to be a suspicious person.

  Also, in this image monitoring system, processing is performed for the person in the normal monitoring area that is not the detail monitoring area, by analyzing the moving features such as the moving speed without performing analysis of individual features such as matching of face images. We consider the privacy while reducing the amount.

JP, 2015-888816, A

  However, even in an area where surveillance is intensified, the majority of the persons present in the area are not suspicious persons, so there is a problem that many unnecessary collation processes occur. Therefore, there is room for improvement in the reduction of the processing amount of the matching process and the consideration for privacy.

  An object of the present invention is to provide a suspicious person detection device capable of detecting a suspicious person in consideration of privacy while reducing the processing amount of collation processing in an area where a large number of unspecified persons gather.

  The present invention detects a person or a plurality of persons from a plurality of images arranged in time series in which a predetermined area is photographed, and tracks a person area of each of the persons among the plurality of images; A face detection unit that detects a face image of the person from the person area in a plurality of images and determines whether the detected face image type is a hidden face image or a bare face image; and a face image by the face detection unit A face image storage unit that distinguishes and stores the bare face image detected by the face detection unit for each person, a suspicious person information storage unit that stores a facial feature amount of a suspicious person, and the person Out of the plurality of images, for the face state change person in which both the concealed face image and the bare face image are detected, the feature amount of the bare face image and the face feature amount of the suspicious person are compared and the matching result is output And said face state change person The face state immutabilization person is outside of a person, a suspicious person detection apparatus characterized by comprising, a face image collation unit which does not perform collation between the face feature amount of the suspicious person.

  Desirably, the face image storage unit further distinguishes and stores the concealed face image detected by the face detection unit for each person, and the face image comparison unit stores the face state change person as the comparison result. Outputting the concealed face image and the bare face image.

  Desirably, the suspicious person information storage unit stores a face image of the suspicious person in association with the face feature amount of the suspicious person, and the face image comparison unit matches the face state change person as the comparison result. And outputting a face image of the match-matched suspicious person.

  According to the present invention, it is possible to provide a suspicious person detection device capable of detecting a suspicious person in consideration of privacy while reducing the processing amount of collation processing in an area where many unspecified persons gather.

BRIEF DESCRIPTION OF THE DRAWINGS It is the structure schematic of the suspicious person detection system which concerns on this embodiment. It is a conceptual diagram which shows a mode that a face image is detected from a time-sequential image, and the face state of each face image is detected. It is a conceptual diagram which shows the example of the data structure of a face image log. It is a flow chart which shows a flow of processing of a suspicious person detection device concerning this embodiment. It is a flowchart which shows the flow of determination processing of the kind of face image.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a schematic configuration diagram of a suspicious person detection system according to the present embodiment. The suspicious person detection system according to the present embodiment is configured to include the camera 10 and the suspicious person detection device 12.

  The camera 10 is configured of a CCD image sensor or a CMOS image sensor. The camera 10 is installed toward a predetermined area, that is, images a predetermined area. The predetermined area in the present embodiment is an area where a large number of unspecified persons gather, and in particular, an area where it is necessary to monitor a suspicious person, for example, an airport, a station, an event hall or the like.

  The camera 10 acquires a plurality of images arranged in time series (hereinafter referred to as “time series images”). The camera 10 may acquire a time-series image by photographing a still image at predetermined intervals (for example, an interval of 200 ms), or may acquire a time-series image by photographing a moving image. . When a moving image is taken, each frame image or each image of several frame intervals constitutes a time-series image.

  The camera 10 has a communication unit (not shown in FIG. 1), and sequentially transmits each image included in the time-series image to the suspicious person detection device 12 via a communication line such as the Internet or LAN. In addition, although the camera 10 may be comprised by one camera, it can also be comprised by two or more cameras. In this case, each of the plurality of images arranged in time series is composed of a plurality of images captured at substantially the same time by a plurality of cameras.

  The suspicious person detection device 12 is constituted by, for example, a computer installed in a central monitoring room installed in an airport, a station, an event hall or the like. The suspicious person detection device 12 may be any device as long as it can exhibit the function described below.

  The communication unit 20 includes, for example, a network adapter. The communication unit 20 exhibits a function of communicating with the camera 10 via a communication line. Specifically, the communication unit 20 sequentially receives the time-series images acquired by the camera 10.

  The display unit 22 includes, for example, a liquid crystal display. The display unit 22 displays a time-series image acquired by the camera 10 in real time. Alternatively, the display unit 22 displays a detection result of a suspicious person from a time-series image by the control unit 30 described later.

  The storage unit 24 is configured to include a semiconductor memory such as a ROM or a RAM, or a magnetic storage medium such as a hard disk. The storage unit 24 stores a suspicious person detection program for operating the respective units of the suspicious person detection device 12. Further, as shown in FIG. 1, the storage unit 24 stores a suspicious person information DB 26 and a face image log 28.

  The suspicious person information DB 26 stores facial feature amounts of the suspicious person. Therefore, the storage unit 24 also functions as a suspicious person information storage unit. The face feature amount of the suspicious person includes, for example, a feature point extracted from the face image of the suspicious person and / or a feature amount in area units. The feature point as the face feature amount may be, for example, a feature point extracted from the bare face of the suspicious person. The facial feature amounts of the suspicious person are provided in advance and stored in the suspicious person information DB 26. Of course, the suspicious person information DB 26 may store face feature amounts of a plurality of suspicious persons.

  Further, the suspicious person information DB 26 may store the face image of the suspicious person in association with the face feature amount of the suspicious person. The face feature amount is information in which the observer can not visually recognize the suspicious person's face, and the face image is information in which the observer can visually recognize the suspicious person's face.

  The face image log 28 stores, for example, processing results by the control unit 30 described later on the time-series images acquired by the camera 10. Details of data stored in the face image log 28 will be described later.

  The control unit 30 includes, for example, a CPU and an image processing IC. The control unit 30 controls each unit of the suspicious person detection device 12 in accordance with the suspicious person detection program stored in the storage unit 24. Further, as shown in FIG. 1, the control unit 30 also functions as a person tracking unit 32, a face detection unit 34, a face image comparison unit 36, and an output processing unit 38.

  The person tracking unit 32 detects one or more persons appearing in the area from each of a plurality of images included in a time-series image in which a predetermined area is photographed, and is an area in which a person is photographed for each person. Set the person area.

  Specifically, in the present embodiment, the person tracking unit 32 extracts the difference between the target image included in the time-series image and the immediately preceding image that is the image immediately before the target image in the time-series image, Based on whether the size, aspect ratio, or the like is a human-like shape, it is determined whether or not there is a region (that is, a person region) in which a person appears in the image of interest. In addition, as a method of detecting a person area, a background subtraction method which performs difference extraction with a background image photographed in a state where there is no moving object, instead of the person area extraction method by this frame subtraction method It is possible to select various methods such as a method using a learning discriminator (person discriminator) in which a discriminant model of a person's image is learned using C. and unmanned images as learning data.

  Furthermore, the person tracking unit 32 tracks the person region of each person among a plurality of images included in the time-series images. Here, tracking of a person area is a process of specifying a person area corresponding to the same person as a specific person area in an immediately preceding image in a focused image among a plurality of images.

  Specifically, in the present embodiment, the person tracking unit 32 identifies the identity between the person area (person area A) extracted from the image of interest and the person area (person area B) of the immediately preceding image in the person area in the image of interest. The determination is made based on the distance between the position of A and the position of the person area B in the immediately preceding image. The position of the person area may be, for example, the position of the center of gravity or the position of the foot of the person area. If there is a person area B in which the distance from the person area A is less than the threshold in the previous image, the persons included in the person area A and the person area B are considered to be the same person. The same sequence ID as the sequence ID assigned to the person area B is assigned. That is, the person tracking unit 32 adds a series ID to each person area, and the person area to which the same series ID is attached indicates the person area corresponding to the same person. On the other hand, when there is no person area B in which the distance from the person area A is less than the predetermined threshold in the previous image, the person included in the person area A is the person newly entering the shooting area from the image of interest. Assume that a new sequence ID is assigned. The determination of the identity may be performed using an image feature amount such as a color histogram in the person area instead of or in addition to the position of the person area. In this case, the tracking accuracy can be improved by using the image feature quantity of the person area excluding the changeable face area. For example, the image feature quantities of the lower third quarter area are used except for the upper quarter of the person area. In addition, when tracking between cameras that do not share a field of view, the image feature amount is the same for a person region extracted in a past image of a time range required for movement between the cameras without being limited to the previous image. The sex determination may be performed.

  The face detection unit 34 analyzes each person area detected by the person tracking unit 32 in a plurality of images included in the time-series image, and detects a face image of a person included in each person area.

  Specifically, in the present embodiment, the face detection unit 34 detects a face area indicating the face of a person from the human area, cuts out a rectangular area including the face area, and detects it as a face image. Then, the detected face image is stored in the face image log 28 in association with the sequence ID corresponding to the person area.

  Specifically, the face detection unit 34 searches for a face area on the person area using the face shape model prepared in advance and stored in the storage unit 24. Specifically, processing for extracting edge pixels in a human region is performed, and an edge distribution of an elliptical shape similar to the outline shape of the head is detected from an image composed of the extracted edge pixels, and a region surrounded by the edge distribution Is extracted as a face area. Here, an elliptical area having a size (area) equal to or larger than a predetermined extraction criterion may be set as an extraction target. Incidentally, the size of the elliptical area is calculated using the length of the major axis and the length of the minor axis. Alternatively, for example, a discrimination model of an image of a head using various head images including a head in bare face, a head in a face concealed state, a head facing backward, and an image in which the head is not shown as learning data. The face region is extracted using a learning discriminator (head discriminator) which has been learned.

  Furthermore, the face detection unit 34 determines whether the type of face image detected from each person area of each image is a hidden face image, a bare face image, or an undecidable image.

  Here, the concealed face image is a face in a state in which the bare face is concealed to such an extent that it can not be compared with the face feature amount of the suspicious person stored in the suspicious person information DB 26 by the face image comparison unit 36 described later. Means an image. For example, as the concealed face image, a face image in which a predetermined number or more (for example, two or more) of ornaments designated in advance are attached and / or face parts are concealed at a certain rate or more (for example, half or more) Can be mentioned. The accessories are, for example, masks, sunglasses, hats and the like.

  Further, the bare face image means a face image in a state in which the bare face is exposed to such an extent that the face image comparing unit 36 described later can compare with the suspicious person's face feature stored in the suspicious person information DB 26. . The glasses and eyebrows may be regarded as part of the bare face.

  Further, the undecidable image means an image that can not be identified whether the face detection unit 34 is a hidden face image or a bare face image. Examples of the undecidable image include an image representing the back of the head of a person.

  The determination process of the type of face image by the face detection unit 34 is executed, for example, as follows. First, the face detection unit 34 determines whether the face image is forward. As the forward determination process, for example, an edge distribution similar to the shape of a part constituting a face such as a forehead, an eye, a nose, a mouth, and a chin is detected from a face image. If the rough positional relationship is similar to the face shape model, it can be determined to be forward. If this is not the case, for example, it can be determined that the face is facing backward if a part constituting the face such as the forehead, eyes, nose, mouth, and chin is not detected from the face image. Alternatively, for example, a learning discriminator (a face direction in which a discrimination model of an image for each face direction is learned using a pair of an image including faces facing various directions and information indicating the direction of the face as learning data) A classifier may be used to determine whether the face image is forward or backward. When the face detection unit 34 determines that the face image is backward, the face detection unit 34 determines that the face image is an undeterminable image.

  If it is determined that the face image is forward, then the face detection unit 34 performs an ornament detection process of detecting an ornament from the face image. The face detection unit 34 uses, as learning data, a set of face images in the state of wearing various accessories such as a mask and sunglasses and information indicating the worn accessories as learning data, and for each combination of accessories and / or for each combination of accessories. By inputting a face image to a learning discriminator (decorative article discriminator) in which a discriminant model of an image is learned, it is possible to detect a decorative article from the face image. The accessory identifier may be, for example, a support vector machine or a learning identifier modeled by one or more convolutional neural networks.

  If the face detection unit 34 detects a predetermined number or more (two or more in the present embodiment) of ornaments from the face image by the ornament detection process, the face detection unit 34 determines that the face image is a hidden face image. When a decorative item is detected or no decorative item is detected, the face image is determined to be a bare face image.

  FIG. 2 is a diagram showing processing results of the person tracking unit 32 and the face detection unit 34 on the images 40 a to 40 c included in the time-series image 40. In the example of FIG. 2, the person tracking unit 32 detects and tracks the person area 42-1 related to the person 1 in each of the images 40 a to 40 c, and detects and tracks the person area 42-2 related to the person 2. A person area 42-3 has been detected and tracked.

  Further, the face detection unit 34 detects face images 44a to 44c of the person 1 from the person area 42-1 of each of the images 40a to 40c, and a face image 46a of the person 2 from the person area 42-2 of each of the images 40a to 40c. To 46c are detected, and face images 48a to 48c of the person 3 are detected from the person area 42-3 of each of the images 40a to 40c.

  Furthermore, the face detection unit 34 determines the types of the face images 44a to 44c, the face images 46a to 46c, and the face images 48a to 48c. In the example of FIG. 2, the face images 44a and 44b of the person 1 are bare face images, and the face image 44c is a hidden face image. The face images 46a and 46c of the person 2 are bare face images, and the face image 46b is an undecidable image. Further, face images 48a to 48c of the person 3 are all concealed face images.

  Processing results of the person tracking unit 32 and the face detection unit 34 are stored in the face image log 28. Specifically, the face image log 28 stores the determination result of the type of the face image detected from each image included in the time-series image 40 in distinction for each series ID (that is, for each person).

  FIG. 3 is a view showing an example of the data structure of the face image log 28. As shown in FIG. The example of FIG. 3 is the processing result of the person tracking unit 32 and the face detection unit 34 with respect to the time-series image 40 illustrated in FIG. 2. The face image log 28 can be said to indicate the history of the type of face image of each person in each image included in the time-series image 40. For example, with regard to the sequence ID 1 (person 1), it is shown that up to the image immediately before the image 40c was a bare face image, but the image 40c has changed to a hidden face image. Further, with regard to the sequence ID 2 (person 2), although the image 40 b has become an undecidable image once, it is shown that the others are bare-face images. In addition, the sequence ID 3 (person 3) is consistently shown to be a hidden face image.

  Further, the face image log 28 stores at least the raw face image detected by the face detection unit 34 from the time-series image 40. In the example of FIG. 2, face images 44a, 44b, 46a and 46c are stored. Each bare face image is stored separately for each sequence ID, that is, for each person. Preferably, not only the raw face image but also the concealed face image (face images 44c and 48a to 48c in the example of FIG. 2) are stored in the face image log 28. The hidden face image is also stored separately for each sequence ID, that is, for each person. As described above, the storage unit 24 also functions as a face image storage unit.

  Returning to FIG. 1, the face image matching unit 36 matches the feature amount of the bare face image detected from the time-series image 40 by the face detection unit 34 with the face feature amount of the suspicious person stored in the suspicious person information DB 26. Execute the matching process. In particular, in the present embodiment, the face image matching unit 36 does not execute the matching process on all the raw face images detected from the time-series image 40, but both the hidden face image and the raw face image from the time-series image 40. The matching process is performed only for the face state change person who is the person who is detected. In other words, the face image matching unit 36 performs the matching process on a face state invariant person who is a person for whom both the concealed face image and the bare face image are not detected from the plurality of images included in the time-series image 40 do not do.

  Here, as the face state non-changing person, a person in which only one of the hidden face image and the bare face image is detected from a plurality of images included in the time-series image 40, and a person in which the hidden face image and the undecidable image are detected. Or, it includes a person whose bare face image and undecidable image are detected.

  The face image matching unit 36 refers to the face image log 28 to identify a series ID (that is, a person) to be subjected to the matching process. In the example of FIG. 3, the face image collation unit 36 targets the series ID 1 (person 1) that is a person with a face state change as a target of the collation process, and the series ID 2 (person 2) and a line ID 3 that is a person with no face state change. The (person 3) is not subjected to the matching process.

  This is because, in the monitoring area which is the shooting area of the camera 10, when the bare-faced person becomes a hidden face from the timing when there is a bare-faced person, or when the bare-faced person becomes a bare-face, the person is a suspicious person It is based on the finding that the possibility of being That is, in the present embodiment, the feature amount of the bare face image of the person is collated with the face feature amount of the suspicious person, focusing on the person who is likely to be a suspicious person.

  In addition, in a plurality of images included in the time-series image 40, it can be said that there is a high possibility that a person in which only the concealed face image is detected or a person in which the concealed face image and the undecidable image are detected is a suspicious person. However, in the concealed face image or the undecidable image, since the matching process can not be performed with the face feature amount of the suspicious person, in the present embodiment, the matching process is not performed for such a person. However, such a person is labeled as a person who needs to be monitored and tracking is continued, and if a bare face image of the person is detected later, the feature amount of the bare face image and the face feature of the suspicious person Make a match between the quantities.

  Among the face feature amounts of a plurality of suspicious persons stored in the suspicious person information DB 26, the face image matching unit 36 has a face status changing person whose similarity to the feature amount of the bare face image of a person whose face state has changed is equal to or more than a predetermined value In the case where the face state change person is identified as a suspicious person. On the other hand, if there is no face feature amount of a plurality of suspicious persons stored in the suspicious person information DB 26, the degree of similarity with the feature amount of the bare face image of the face state change person is not less than a predetermined value It specifies that the said face state change person is not a suspicious person memorized by suspicious person information DB26. In addition, as a calculation method of the degree of similarity, a known method can be adopted.

  The output processing unit 38 outputs (informs) the result of the matching process. In the present embodiment, the result of the matching process is displayed on the display unit 22. As described above, since the time-series image acquired by the camera 10 is displayed in real time on the display unit 22, the result of the matching process of the face image matching unit 36 is displayed superimposed on the time-series image. It is also good. For example, the person area of the person specified as the suspicious person can be colored, and the person areas of the person who is not the suspicious person can be displayed distinguishably in a display manner such as not coloring. When a suspicious person is detected, in addition to the display on the display unit 22, notification by voice may be performed.

  In addition, the output processing unit 38 is a person who needs monitoring (that is, a person in which only the concealed face image is detected in a plurality of images included in the time-series image 40) It may be made to output what was detected. In such a case, for example, as in the case of the suspicious person, the person area of the person identified as the person in need of monitoring is colored in a color different from that of the suspicious person, and the person area of the person who is not in need of It is possible to distinguishably display both in a display mode such as not. In addition to the display on the display unit 22, a notification by voice may be made even when a person who needs monitoring is detected.

  In addition, preferably, the output processing unit 38 may output, as a matching result, a hidden face image and a bare face image of a face state-changed person detected from the time-series image. The output concealed face image and bare face image are provided, for example, to a monitoring person who monitors the monitoring area. By referring to the provided concealed face image and bare face image, the surveillance staff can more easily search for the person whose face state has changed.

  The output processing unit 38 may output the concealed face image and the bare face image of only the face state change person identified as the suspicious person, but the suspicious person detection device 12 and a double check of the suspicious person by the surveillance staff In order to enable the output processing unit 38, not only the face state change person specified as the suspicious person but also the concealed face image and the bare face image of the face state change person specified as not the suspicious person are output You may

  Further, the output processing unit 38 may output the face image of the suspicious person associated with the face feature amount of the suspicious person who has matched the feature amount of the bare face image of the face state change person as the matching result. The face image of the suspicious person is also provided, for example, to a surveillance person who monitors the surveillance area. The surveillance officer can easily find out the person whose face state has changed by referring to the provided face image of the suspicious person.

  As described above, according to the suspicious person detection device 12 according to the present embodiment, the processing amount of the verification process is reduced because the person with high possibility of being a suspicious person is narrowed down and compared with the face feature amount of the suspicious person. can do. Further, in consideration of the privacy of a person (a person who is unlikely to be a suspicious person) included in the time-series image 40, a suspicious person can be detected.

  Hereinafter, the flow of processing of the suspicious person detection device 12 according to the present embodiment will be described with reference to the flowchart of FIG. 4. The flowchart of FIG. 4 shows the flow of processing executed when one image included in the time-series image is received from the camera 10. That is, in the present embodiment, the suspicious person detection device 12 executes the process of the flowchart of FIG. 4 each time an image is received from the camera 10.

  In step S10, the suspicious person detection device 12 acquires an image (one of time-series images) captured by the camera 10. Here, it is assumed that the suspicious person detection device 12 has already acquired a plurality of images included in the time-series image.

  In step S12, the person tracking unit 32 detects one or more person regions from the image (the image of interest) acquired in step S10, and performs tracking processing on each of the detected person regions. That is, the person tracking unit 32 executes a process of giving a series ID to each of the detected person areas.

  The processes in step S14 and the subsequent steps are performed for each person (person area) detected in step S12.

  In step S14, the face detection unit 34 identifies a notable person region from the plurality of person regions detected in step S12, and detects a face image of the notable person from within the notable person region. Then, the type of the face image is determined. That is, it is determined whether the face image of the person of interest is a hidden face image, a bare face image, or an undecidable image.

  In step S16, the person tracking unit 32 and the face detection unit 34 detect the result of person detection and tracking processing (sequence ID attached to the attention person area) and the determination result of the type of the face image of the attention person as the face image log Store in 28.

  In step S18, the person tracking unit 32 determines whether the attention person area is in the tracking state. In the present embodiment, for each series ID, the person tracking unit 32 has the same sequence ID of the noted person region as that of any person region included in the image past the noted image (ie, the noted person is in the past If the attention person area is determined to be in the tracking state and the series ID of the attention person area is an ID newly assigned to the attention image (ie, the attention person is new from the attention image) When it is reflected on the screen, it is determined that the attention person area is not in the tracking state.

  If the focused person area is not in the tracking state, the process proceeds to step S20, and in step S20, the focused person area is set in the tracking state.

  In step S22, the face detection unit 34 determines whether the type of the face image of the person of interest determined in step S14 is a hidden face image. If the face image is a hidden face image, the process proceeds to step S24. In step S24, the face detection unit 34 sets the watched person included in the watched person area to the watched person (this setting is stored in the face image log 28), and the output processing unit 38 detects the watched person. Report that it was done. If the face image is not a hidden face image, that is, if it is a bare face image or an undecidable image, no processing is performed and the processing relating to the target person area is ended. Thereafter, the processing from step S14 is repeated for other person regions included in the image of interest.

  Returning to step S18, if it is determined that the attention person area is in the tracking state, the process proceeds to step S26, and in step S26, the person tracking unit 32 determines whether the attention person is a person who needs monitoring. If the concealed face image of the person of interest is detected in the plurality of images acquired in the past, the person of interest is set as the person who needs to be monitored, so the process proceeds to step S28.

  In step S28, the face detection unit 34 determines whether the type of face image determined in step S14 is a bare face image. If the face image is not a bare face image, that is, the face image of the noted person is a hidden face image or an undecidable image in a plurality of images acquired in the past, and the face image of the noted person continues to be a hidden face image even in the noted image Alternatively, if the image is an undecidable image, the process related to the focused person area is ended without performing any process. Thereafter, the processing from step S14 is repeated for other person regions included in the image of interest.

  If it is determined in step S28 that the face image is a bare face image, that is, the face images of the noted person in the plurality of images acquired in the past are the concealed face image or the undecidable image, but the noted person in the noted image If the face image of has changed to a bare face image, the process proceeds to step S30.

  In step S30, the face image matching unit 36 performs matching processing between the feature amount of the bare face image acquired in step S14 and the face feature amounts of a plurality of suspicious persons included in the suspicious person information DB 26.

  In step S32, the face image matching unit 36 determines whether the face feature amounts of the plurality of suspicious persons included in the suspicious person information DB 26 have a face feature amount that matches the feature amount of the bare face image acquired in step S14. It is determined whether or not. If there is a match, the process proceeds to step S34, and in step S34, the output processing unit 38 reports the person of interest as a suspicious person. On the other hand, if there is no match, the process on the person-of-interest area is ended without performing any process. Thereafter, the processing from step S14 is repeated for other person regions included in the image of interest. If the match does not match in step S32, it may be specified that the person of interest is not a suspicious person and may be excluded from the candidates for the matching process thereafter.

  Returning to step S26, if the focused person is not the person who needs to be monitored, that is, if the concealed face images of the focused person are not detected in a plurality of images acquired in the past, the process proceeds to step S36.

  In step S36, the face detection unit 34 determines whether the type of face image determined in step S14 is a hidden face image. If the face image is not a concealed face image, that is, the face image of the noted person is a bare face image or an undecidable image in a plurality of images acquired in the past, and the face image of the noted person continues a bare face image or even in the noted image. If the image is an undecidable image, the process related to the person-of-interest area is ended without performing any process. Thereafter, the processing from step S14 is repeated for other person regions included in the image of interest.

  When it is determined in step S36 that the face image is a concealed face image, that is, the face images of the noted person in the plurality of images acquired in the past are bare face images or undecidable images, but the noted person in the noted image If the face image of has changed to a hidden face image, the process proceeds to step S38.

  In step S38, the face detection unit 34 refers to the face image log 28 and refers to the determination result of the type of the face image of the person of interest in one of the plurality of images acquired in the past. If the determination result of the face image of the noted person in the image is not a bare face image, whether the determined result of the face image of the noted person in the other images of the plurality of images acquired in the past is a bare face image in step S38 again It is determined whether or not. As described above, the face detection unit 34 searches the plurality of images acquired in the past for an image determined that the face image of the person of interest is a bare face image.

  If an image in which the face image of the person of interest is determined to be a bare face image is detected from the images acquired in the past, the process proceeds to step S40, and in step S40, the face image comparison unit 36 determines in step S38. The bare face image (this is stored in the face image log 28) of the noted person detected from the captured image is acquired, and the feature amount of the bare face image and the faces of a plurality of suspicious persons included in the suspicious person information DB 26 A matching process is performed on each of the feature quantities.

  In step S42, the face image matching unit 36 determines whether the face feature amounts of the plurality of suspicious persons included in the suspicious person information DB 26 have a face feature amount that matches the feature amount of the bare face image obtained in step S40. It is determined whether or not. If the collation matches, the process proceeds to step S44, and the output processing unit 38 notifies the person of interest as a suspicious person.

  On the other hand, when it is determined in step S42 that the matching is not determined, the process returns to the loop of step S38 again, and the face image log 28 is referred to, and the face image of the person of interest is selected from a plurality of images acquired in the past. It is searched whether there is any other image determined to be a bare face image. This may be the case, for example, because the matching process of the face image matching unit 36 can not be appropriately performed because the condition of the bare face image of the noted person who was the target of the matching process of the previous time was bad. This is in order to search for a better bare face image from a plurality of images.

  If a bare face image of a noted person who matched the suspicious person is not found among the plurality of images acquired in the past, the process proceeds to step S46. In step S46, the face detection unit 34 sets the watched person to the watched person, and the output processing unit 38 notifies that the watched person has been detected.

  Hereinafter, with reference to the flowchart of FIG. 5, the flow of the face state determination process by the face detection unit 34 will be described.

  In step S50, the face detection unit 34 performs a forward determination process on the face image. As described above, the forward determination process is performed by comparing the shape of the part of the face detected from the face image with the face shape model. Alternatively, the determination is performed using a face direction identifier.

  In step S52, the face detection unit 34 determines whether or not the face image is forward as a result of the forward determination process of the face image. If the face image is not forward, for example, if the face image is backward, the process proceeds to step S54, and the face detection unit 34 determines that the face image is an undeterminable image.

  If it is determined in step S52 that the face image is forward, the process proceeds to step S56, and in step S56, the face detection unit 34 performs an ornament detection process on the face image. The ornament detection process is performed using the ornament identifier as described above.

  In step S58, the face detection unit 34 determines whether the face image is a hidden face image based on the result of the ornament detection process. In the present embodiment, when two or more ornaments are detected from the face image, it is determined that the face image is in the concealed state, and the process proceeds to step S60. In step S60, the face detection unit 34 determines that the face image is a hidden face image.

  If one decorative item is detected from the face image or no decorative item is detected from the face image, it is determined that the face image is not in the concealed state, and the process proceeds to step S62. In step S62, the face detection unit 34 determines that the face image is a bare face image.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is not limited to the said embodiment, A various change is possible unless it deviates from the meaning of this invention.

Reference Signs List 10 camera 12 suspicious person detection device 20 communication unit 22 display unit 24 storage unit 26 suspicious person information DB 28 face image log 30 control unit 32 person tracking unit 34 face detection unit 36 face image collation Part, 38 output processing part.

Claims (3)

A person tracking unit that detects one or a plurality of persons from a plurality of images arranged in time series in which a predetermined area is photographed, and tracks a person area of each of the persons among the plurality of images;
A face detection unit that detects a face image of the person from the person area in the plurality of images, and determines whether the detected face image is a hidden face image or a bare face image;
A face image storage unit that stores the result of determination of the type of face image by the face detection unit and the bare face image detected by the face detection unit for each of the persons;
A suspicious person information storage unit that stores a facial feature amount of the suspicious person;
Among the persons, for a face state change person in which both the hidden face image and the bare face image are detected from the plurality of images, the feature amount of the bare face image is compared with the face feature amount of the suspicious person A face image comparing unit that outputs the face state non-changing person who is a person other than the face state changing person, and does not check the face feature amount of the suspicious person;
An apparatus for detecting a suspicious person, comprising: The face image storage unit further stores the concealed face image detected by the face detection unit separately for each person.
The face image matching unit outputs the hidden face image and the bare face image of the face state change person as the matching result.
The suspicious person detection device according to claim 1, characterized in that: The suspicious person information storage unit stores a face image of the suspicious person in association with the face feature amount of the suspicious person,
The face image matching unit outputs, as the matching result, a face image of a suspicious person who has matched the face state change person.
The suspicious person detection device according to claim 1 or 2, characterized in that.