JP6739459B2 - Person matching device - Google Patents

Description

The present invention relates to a person matching device, and more particularly to a person matching device that determines whether or not a target person is photographed in an input person image.

Biometric information matching technology such as face images reduces both the acceptance of others who authenticate other people other than pre-registered registrants as the registrant and the rejection of the registrant who fails to authenticate the registrant himself. Is required. Patent Document 1 discloses, for face image matching, extracting the feature amount so as to utilize fine structures such as moles, stains, and wrinkles for the purpose of improving matching accuracy.

JP, 2017-182606, A

In the face image matching technique, it is known that the image quality, which is the degree of image quality such as high and low resolution and contrast, greatly affects the degree of similarity between the registered face image and the input face image.

Here, in a surveillance camera system that applies face image matching technology to detect a pre-registered suspicious person from the surveillance image, the photographed person who seems to be a suspicious person and the camera are separated, and the face part is small and pixel There may be a small number, or the shadow of a hat or a building may hang on the face and the contrast of the face part may be insufficient. This is, so to speak, a state in which the image quality of the face image is low in face image matching, and it is difficult to accurately extract the feature amount related to the fine structure of the face because the face is unclearly reflected. Then, the lowering of the degree of similarity calculated using the feature amount of the fine structure affects the detection performance, which may cause a false alarm, that is, a suspicious person may miss it.

On the other hand, it is not always possible to obtain a face image of a suspicious person clearly captured from the front, and there is no choice but to use a face image automatically or manually extracted from past monitoring images as a registered person image. Then, a registrant with a high image quality of the registered person image (a suspicious person) and a registrant with a low image quality (a suspicious person) coexist, but the latter has a low degree of similarity calculated using the feature amount of the fine structure. Therefore, if the threshold value is set low in order to prevent the suspicious person from overlooking in consideration of this, there is a fear that a person who is not a suspicious person is determined to be a suspicious person and an erroneous alarm is issued to issue an unnecessary warning.

It is an object of the present invention to provide a person matching device that can accurately determine a person.

In order to achieve the above-mentioned object, a person collation device according to the present invention comprises an image acquisition means for acquiring a photographed image of a predetermined area, a detection means for detecting an input person image from the photographed image, and a person. A coarse structure feature amount representing an image feature relating to a rough structure and a fine structure feature amount representing an image feature relating to a fine fine structure in a person, the rough structure feature being extracted from a registered person image of a target person. Amount and the fine structure feature amount are stored for each of the target persons, quality evaluation means for evaluating an input person image quality indicating the degree of quality of the input person image, and the rough structure from the input person image. A coarse structure matching unit that extracts a feature amount and calculates a coarse structure matching degree with a rough structure feature amount stored in the storage unit; and a storage unit that extracts the fine structure feature amount from the input person image. A fine structure matching means for calculating a fine structure matching degree with a fine structure feature amount stored in, and it is determined whether or not the target person is photographed in the input person image using at least the rough structure matching degree. The person determination means for performing the rough structure matching, wherein the person determination means determines that both the input person image quality and the registered person image quality indicating the degree of quality of the registered person image are high quality equal to or higher than an image quality threshold value. It is characterized in that whether or not the target person is photographed in the input person image is determined using the degree and the degree of fine structure matching.

In the person determination means according to the present invention, when both the input person image quality and the registered person image quality are the high quality, the coarse structure coincidence is equal to or higher than a first threshold and the fine structure coincidence is the first degree. When it is determined that the target person is photographed in the input person image when it is two thresholds or more, the input person image quality is the high quality, but the registered person image is low quality less than the image quality threshold value. Determines that the target person is captured in the input person image when the coarse structure matching degree is equal to or higher than a third threshold value higher than the first threshold value.

The person determination means according to the present invention calculates an integrated coincidence degree that integrates the coarse structure coincidence degree and the fine structure coincidence degree when both the input person image quality and the registered person image quality are the high quality. , It is determined whether or not the target person is photographed in the input person image based on the integrated coincidence degree, and the input person image quality is the high quality, but the registered person image is low quality less than the image quality threshold value. In this case, it is determined from the rough structure matching degree whether or not the target person is photographed in the input person image.

The person determination means according to the present invention, when both the input person image quality and the registered person image quality are the high quality, the higher the registered person image quality, the higher the contribution of the fine structure matching degree to the integrated matching degree. Then, the integrated coincidence degree is calculated.

The person matching apparatus according to the present invention further traces the input person images detected from the plurality of captured images sequentially acquired for the same person, and generates series data in which the traceable input person images are associated with each other. A tracking means, and the person determination means, for each of the series data, the highest input person image quality among the input person image quality of the input person image included in the series data is a low quality less than the image quality threshold value. If there is, it is determined that the target person is photographed in the input person image included in the series data when the degree of coarse structure matching is equal to or higher than the third threshold that is higher than the first threshold.

As described above, according to the person matching device of the present invention, it is possible to obtain the effect that the person determination can be performed accurately.

It is a schematic diagram which shows a mode that the image monitoring apparatus which concerns on embodiment of this invention is installed in the monitoring area|region, and is operating. It is a block diagram which shows the structure of the image monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows an example of registration data. (A) It is a figure which shows an example of a sharp face image, and (b) It is a figure which shows an example of a defocused face image. It is a figure which shows an example of series data. It is a figure which shows an example of the change of the input person image quality of series data. 5 is a flowchart showing an operation of image monitoring processing by the image monitoring device according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case will be described as an example in which the present invention is applied to a face matching device and a person between an input person image and a registered person image is matched.

<System configuration>
FIG. 1 schematically shows a state in which the image monitoring apparatus according to the present embodiment is installed and operated in the monitoring area 100.

As shown in FIG. 1, a person is photographed in the monitoring area 100, and when a suspicious person is included therein, the person is notified to the outside, for example, a security center.

FIG. 1 shows a person 101, a person 102, and an imaging device (imaging means) 2 for photographing the two persons.

FIG. 2 schematically shows the configuration of the image monitoring device 20 in this embodiment as a block diagram.

The image pickup device 2 is a surveillance camera, and is preferably installed slightly downward from the ceiling as shown in FIG. A plurality of lenses may be prepared, but it is preferable to use a wide-angle lens rather than a telephoto lens to secure the field of view.

When the notification device 3 receives a signal indicating the presence of a suspicious person from the face matching device 1, the notification device 3 sounds an alarm sound or lights an indicator light. Alternatively, the notification device 3 may be installed in the security center and receive the signal from the output unit 13 via the network.

The face matching device 1 includes an image acquisition unit 10, an image processing unit 11, a storage unit 12, and an output unit 13. The face matching device 1 is an example of a person matching device.

The image acquisition unit 10 (image acquisition means) is an I/F circuit that receives an input image from the imaging device 2.

The output unit 13 is an I/F circuit that outputs the presence of a suspicious person from the image processing unit 11 to an external device or the notification device 3.

The storage unit 12 has a semiconductor memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 12 stores a computer program for controlling the face matching device 1 and various data, and inputs/outputs these pieces of information with the image processing unit 11. The various data includes registrant data 121 including a person image of a suspicious person who has received information from a person who has an appropriate authority in advance, and series data 122 of the same person summarized by a tracking unit 114 described later. Is included.

The contents of the registrant data 121 will be described with reference to FIG.

FIG. 3 schematically shows the contents of the registrant data 121 in the form of a table.

The registrant data 121 includes an ID 1211, a registered person image 1212, a coarse structure feature amount 1213, a fine structure feature amount 1214, and a registered person image quality 1215. In addition, other factors such as a name and a reason for determining a suspicious person, if known, may be added depending on individual specific circumstances.

The ID 1211 is an identification number uniquely assigned to each suspicious individual. Multiple person images may be registered for one suspicious person. In that case, a branch number may be added to indicate that a plurality of person images are registered for the same suspicious person (for example, “150-1” and “150-2” with respect to the ID 1211 of “150”). It is also possible to separately give an ID for the image and an ID for the person image and associate them.

The registered person image 1212 is a face image manually extracted by the administrator of the image monitoring device 20 from the monitoring image at that time for each suspicious person, for example, a person who shoplifted in the past, or the image processing unit 11 described later. It is a face image extracted by the face detection means 111.

Since the registered person image 1212 itself does not have to be directly used for the operation of the image monitoring apparatus 20 in this embodiment, a coarse structure feature amount 1213, a fine structure feature amount 1214, and a registered person image quality 1215, which will be described later, are determined. After that, it may be deleted without being remembered.

The rough structure feature amount 1213 is a feature amount of the registered person image 1212 extracted by the rough structure matching unit 115 in the work at the time of registration. Various kinds of feature amounts are well known in the field of image matching and may be appropriately adopted. For example, a HoG feature amount obtained by dividing a human image into small regions or a local region around the extracted feature points. The wavelet feature amount obtained from the above may be used, and these feature amounts may be stored as a feature amount vector. Since it is difficult to illustrate the feature amount itself, “*” is used instead in FIG.

The fine structure feature amount 1214 is a feature amount of the registered person image 1212 extracted by the fine structure matching unit 116 during the work at the time of registration. The feature amount is a feature amount obtained by referring to minute features of the skin such as spots, moles, and wrinkles, and may be obtained by following the method disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2017-182606). It may be stored as a feature vector. Since it is difficult to illustrate the feature amount itself, “*” is used instead in FIG.

The registered person image quality 1215 is the image quality of the registered person image 1212 evaluated by the quality evaluation unit 113 in the work at the time of registration. The administrator in charge of registration work may make the decision by visual confirmation.

As the registered person image quality 1215, as shown in FIG. 3, a number obtained by determining the image quality and leveling is stored. Evaluation values regarding other image quality may be directly stored. Alternatively, when the image quality evaluation has three levels, the character strings of “high”, “medium”, and “low” may be stored.

For example, it is assumed that the lowest level 0 is completely useless for collation and the highest level 10 is high quality like an ID photograph.

In FIG. 3, the registered person image 1212 with the ID 1211 of “101” has a condition such as sharpness and brightness that is suitable for the matching process. Therefore, the quality evaluation means 113 evaluates the registered person image 1215 as the registered person image quality 1215. A certain "level 9" is stored.

Similarly, since the sharpness of the registered person image 1212 having the ID 1211 of "102" is somewhat unsuitable for the collation processing, "level 6" which is the result of evaluation by the quality evaluation means 113 as the registered person image quality 1215. Is remembered.

Similarly, the registered person image 1212 with the ID 1211 of "103" is in a state in which the sharpness can barely be subjected to the collation processing, and therefore, as the registered person image quality 1215, "level 3" which is the result determined by the quality evaluation unit 113 is Remembered

Similarly, the registered person image 1212 having the ID 1211 of “104” has a contrast that is slightly unsuitable for the matching process, and therefore the registered person image quality 1215 is “level 7” which is the result evaluated by the quality evaluation unit 113. Remembered

Returning to FIG. 2, the face detection unit 111 detects a face image as an input person image from the input image and outputs it to the feature point extraction unit 112.

A known method may be used to detect the face image. For example, a method of extracting a face image with a filter called a discriminator that has been learned, or generating a binarized edge image of an input image, and defining an elliptical portion that is the shape of the face in the edge image as a face region, May be the input person image.

Further, when a face image of a suspicious individual is registered as a registered person image, the face image of the suspicious individual has a function of obtaining the average brightness and the variance of the brightness and outputting the result to the quality evaluation unit 113 as the contrast of the face image.

The feature point extraction unit 112 extracts a plurality of face feature points corresponding to respective parts such as eyes, nose, mouth, eyebrows, and face outlines from the input person image extracted by the face detection unit 111, and the input person image. It is a means for obtaining the upper position coordinates. Each part such as eyes, nose, mouth, eyebrows, and face contour is an example of the rough structure.

Many methods for extracting facial feature points have been proposed in the past, and known methods may be adopted as appropriate. For example, a method of extracting an image around a facial feature point with a filter called a discriminator, or generating a binarized edge image of an input image and detecting a characteristic shape around the facial feature point in the edge image The method of doing is good.

Further, when the face image of the suspicious individual is registered as the registered person image, the coordinate information of the feature points extracted for the face image is output to the quality evaluation unit 113 as data for evaluating the image quality of the registered person image. Have a function.

The quality evaluation unit 113 evaluates the image quality, which is the quality level of the registered person image, based on the data output from the face detection unit 111 and the feature point extraction unit 112 during the registration work, and the registered person data 121 in the storage unit 12. The registered person image quality 1215 of FIG. In addition, the image quality of the input person image acquired during monitoring is evaluated each time.

It will be described using the schematic diagram shown in FIG. 4 that the level of the image quality affects whether or not the fine feature can be utilized in the matching process.

A sharp face image 300 is shown in FIG. As it is sharp, large parts such as eyes and mouth that are used for normal face matching, as well as fine structures such as wrinkles 303 on the outer corners of the eyes, spots 301 on the face, and moles 302 are clearly shown, and as information showing individuality. It can be fully utilized.

FIG. 4B shows a face image 310 with a slight blur. Even if large parts such as eyes and mouth can be used for face matching as usual, fine structures such as wrinkles 313 on the outer corners of the eyes, spots 311 on the face, and moles 312 will be blurred. It is also difficult to use the fine features of for matching.

In the above description, the presence or absence of sharpness is described, but in reality, the resolution of the face image in the input image, that is, the size, becomes a problem. This is because in image monitoring, the size of a person present in the field of view cannot be assumed in advance, and the face portion may only appear small. On the other hand, in the collation processing, so-called normalization processing is performed in order to make the processing uniform in size, but the face image that appears only small is enlarged. In that case, the enlarged result will be a blurred face image as shown in FIG. Particularly in the field of image monitoring, since it is common to use a wide-angle lens to target a wide range with a small number of cameras, normalization often results in a blurred face image.

Regarding the contrast, it is difficult to use the information of the fine structure from the face image acquired under the suitable lighting condition, and it is difficult to use the information under the dark condition. The quality of the image, such as the presence/absence of sharpness, affects the matching process, but if the matching process is performed without considering the quality of the image, the accuracy may be deteriorated.

Therefore, when the registered person image of the suspicious person is registered, the quality evaluation means 113 evaluates the image quality of the registered person image based on the data output from the face detection means 111 and the feature point extraction means 112, and sets it as the registered person data 121. Remember. The image quality of the input person image is evaluated each time it is obtained and stored in the series data described later.

The quality evaluation unit 113 estimates, for example, the resolution of the registered person image as the registered person image quality, and if the resolution is equal to or higher than a predetermined value, the quality evaluation unit 113 determines that the quality is high and the level is 8 to 10 depending on the resolution, and the resolution is higher than the predetermined value. If it is low, medium quality is evaluated as levels 4 to 7 depending on the resolution, and if the resolution is lower, it is evaluated as low quality and levels 0 to 4 are stored according to the resolution and stored in the registered person image quality 1215. That is, if the detected size of the face image in the registered person image is large, the registered person image is evaluated as high quality, and if the detected size of the face image in the registered person image is small, the registered person image is evaluated as low quality.

Here, when the position information of the feature points of the rough structure feature amount is input from the feature point extraction unit 112, the quality evaluation unit 113 selects, for example, the number of pixels between the center of the right eye and the center of the left eye among the feature points. The (eye distance) may be counted and used to estimate the resolution. That is, if the number of pixels between the characteristic points is equal to or greater than a certain level, the quality is high and the level is 8 to 10 in accordance with the distance between eyes. If the number is smaller, the quality is evaluated as levels 0 to 3 and stored in the registered person image quality 1215.

It is preferable to use the pixel value between feature points such as the distance between eyes to estimate the resolution when detecting a face, for example, when wearing a hat and the color or pattern of the background wall or floor is different from hair or skin. This is because if they are similar, the size of the face may differ from the actual size in the image, and it may be inappropriate to estimate the resolution. In that respect, the center of the eye is often reflected as a light skin color around it, and the contrast with the color of the pupil is clear and stable, and extraction of feature points and estimation of resolution can be expected.

Alternatively, when the information about the average brightness of the registered person image or the distribution of the brightness is input from the face detection unit 111, the quality evaluation unit 113 determines that the quality is high if they are in a range suitable for the matching process. Levels 8 to 10 according to the information, levels 4 to 7 as medium quality if slightly out of the range, and levels 0 to 3 as low quality if significantly out of range, and stored in the registered person image quality 1215. ..

Alternatively, the quality evaluation unit 113 may perform well-known sharpness determination processing on the registered person image to estimate blurring, and evaluate the image quality according to the estimated blurring.

The quality evaluation unit 113 may evaluate the image quality of the registered person image by combining the above evaluation results. For example, when measuring the resolution and the contrast as described above, the number of levels may be averaged.

Alternatively, if each evaluation result is “high”, “medium”, and “low”, “high” is stored in the registered person image quality 1215 when both are “high”, and “high” is stored when both are “low”. “Low” may be stored in the registered person image quality 1215, and in other cases, “medium” may be stored.

Further, the quality evaluation unit 113 evaluates the image quality of the input person, which represents the degree of quality of the input image of the person, in the same manner as the image quality of the registered person. The registered person image quality may be manually input.

The tracking unit 114 is a unit that stores the input person image extracted by the face detection unit 111 at each time as the series data 122 of the storage unit 12 in association with each other for the same person. A known face image tracking process may be appropriately adopted for association. For example, at successive times, the coordinates of the input person image in the input image acquired by the image acquisition unit 10 are referred to, and the barycentric coordinates can be grouped under the condition that they are close to each other by a certain amount or more. Alternatively, it can be tracked by using a method such as an optical flow or a particle filter, and further, the input person images may be directly collated to be tracked as belonging to the same person.

The contents of the series data 122 will be described with reference to the schematic diagram of FIG.

FIG. 5 shows the series data in a tabular format. Each row shows one series of data. One piece of series data means that the face detection means 111 starts tracking for a certain person from the input image, and the tracking means 114 tracks the person until the person ends the tracking because the person is deviated from the input image. This is a data group that does not require further processing because the verification result has been finalized during tracking.

The sequence number 1221 is an ID uniquely assigned to each sequence data.

The input person image 1222 is a face image extracted by the face detection unit 111.

The tracking information 1223 is the coordinate value in the input image for each of the input person images 1222 and the input person image quality determined by the quality evaluation unit 113.

In addition to this, a flag indicating that tracking is in progress, a flag indicating that some kind of notification processing has been performed, and the like may be added as necessary for use in tracking processing.

A method of referring to the input person image quality in the series data will be described with reference to FIG.

In FIG. 6A, the image quality of the input person of the series data whose series number 1221 in FIG. 5 is “12” is schematically shown. In the series data, the input person image quality of the input person image denoted by reference numeral 600 does not exceed the image quality threshold value 620, and is not suitable for the authentication determination process. Therefore, the authentication determination result is not output while the input person images are obtained.

Since the input person image quality of the input person image 610 exceeds the image quality threshold 620, the authentication determination result is determined and output by using this input person image and calculating the degree of coincidence (similarity degree) by means described later. To do.

On the other hand, in FIG. 6B, the image quality of the input person of the series data having the series number 1221 of “25” in FIG. 5 is schematically shown. The series data shows a case where the input person image quality of any of the input person images does not exceed the image quality threshold value 620 and the tracking ends. In this case, all of them are unsuitable for the authentication determination process and in principle do not produce a result, but this may lead to omission (unreporting) of a suspicious person. On the other hand, if the image quality is simply low and the determination threshold is lowered, there is a risk of falsely erroneously determining another person who is not similar to the person.

Therefore, in the means to be described later, the degree of coincidence (similarity) is obtained using the input person image 630 showing the highest input person image quality, and the authentication determination result is obtained.

The coarse structure matching unit 115 characterizes the relative positions of the respective feature points extracted from the input person image and the sizes and colors of the components of the coarse structure of the face such as eyes, nose, mouth, etc. A coarse structure matching degree Sa representing the degree of similarity with the coarse structure feature amount 1213 stored in the data 121 is obtained. As the feature amount, ones well known in the field of face image authentication can be appropriately selected and used, and the feature amount vectors that are arranged in the components are used, but the coarse structure matching degree is set to 0% for the norm of the difference between the feature amount vectors. The maximum value can be set with care and can be the reciprocal.

At the time of registration of the registered person image, the coarse structure feature amount obtained from the registered person image is stored in the coarse structure feature amount 1213 of the registrant data 121.

The fine structure matching unit 116 characterizes the features of the fine structure of the face such as wrinkles extracted from the input person image, and represents the degree of similarity with the fine structure feature amount 1214 stored in the registrant data 121. The fine structure matching degree Sm is obtained. As the feature amount, the one disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2017-182606) may be used as described above, and the feature amount vector that is included in the components is arranged, but the fine structure matching degree is the difference between the feature amount vectors. Regarding the norm of, it is possible to set the maximum value and take the reciprocal while paying attention to zero percent.

At the time of registration of the registered person image, the fine structure feature amount obtained from the registered person image is stored in the fine structure feature amount 1214 of the registrant data 121.

The person determining unit 117 refers to the input person image quality and the registered person image quality 1215 evaluated by the quality evaluating unit 113 and stored in the series data 122, and uses the coarse structure coincidence degree Sa and the fine structure coincidence degree Sm to input. It is determined which registered person image 1212 the person in the person image is a suspicious person.

When the input person image quality is high quality equal to or higher than a predetermined image quality threshold value (corresponding to the input person image 610 in FIG. 6A, for example, “level 7” or higher), the registered person image quality 1215 of the registered person image to be compared is set. It is read from the storage unit 12.

The registered person image quality 1215 is also of high quality, for example, in the case of "level 7" or higher in FIG. 3, both the input person image and the registered person image are of high quality, which is a suitable condition for matching, and therefore the rough structure Both the coincidence degree Sa and the fine structure coincidence degree Sm are used. That is, when the coarse structure matching degree Sa is equal to or higher than the first threshold value and the fine structure matching degree Sm is equal to or higher than the second threshold value, the person determination means 117 shows the person of the registered person image in the input person image. To determine. This is because the information is obtained not only about the part having a rough structure but also about the fine structure such as wrinkles. Therefore, the intention is to prevent the person from rejecting the person by checking the degree of agreement of the fine structure.

When a plurality of registered person images satisfy the person determination condition, the registered person images showing the highest degree of coincidence between the two registered person images are adopted as the matching results. If one registered person image simultaneously satisfies the threshold conditions for the two matching degrees but does not simultaneously show the highest value, the one having the highest fine structure matching degree Sm is adopted. This is based on the idea that it is preferable to attach importance to the similarity of the fine structure that represents individuality, under the condition that the rough structure is sufficiently similar.

Although the input person image quality is high quality, the registered person image quality 1215 of the registered person image to be compared is not high quality. For example, in the case of “level 5” in FIG. Not a suitable condition. Under this condition, even if the feature amount for the fine structure is obtained from the registered person image, necessary information is missing because the registered person image is blurred, so it is inappropriate to use the fine structure matching degree Sm. You can say that. Therefore, the person determination means 117 determines only from the coarse structure coincidence degree Sa, and determines that the person of the registered person image appears in the input person image when the coarse structure coincidence degree Sa is equal to or larger than the third threshold value. To do. However, the third threshold value is set to a value higher than the first threshold value, and the determination condition is made strict. This is an intention to strictly require that face images are more similar to each other than matching under good conditions. For example, regarding the values that the coarse structure matching degree and the fine structure matching degree can take, if the maximum value is grasped in advance by an experiment and 80% thereof is set as the first threshold value, the third threshold value is also 90%. To do.

On the other hand, if the image quality of the input person at a certain time is not high, the condition is not suitable for the matching. In this case, since it is inappropriate to obtain the matching result using the input person image, it is necessary to store the input person image quality as shown in the series data 122 of FIG. 5 and not output the matching result. To do. When the input person image quality included in the series data 122 of the person is less than the image quality threshold value after the tracking of the input person image of the person is completed, the input person having the highest image quality in the series data 122 is obtained. Using the image, the rough structure matching degree Sa and the third threshold value are compared to obtain a matching result. If any of the image quality of the input person exceeds the image quality threshold value, it means that the image quality is high.

Originally, if the image quality of the input person image is not ensured, it may be considered as unsuitable for collation, but in a situation where tracking is performed to some extent, a suspicious person may stay (wander around) in the imaging region. Therefore, in order to prevent a situation in which a suspicious individual is not reported even though the suspicious person is shown (missing report), the collation determination process is performed aggressively.

However, as in the case described above, the third threshold is used as the determination condition based only on the rough structure matching degree.

The integrated coincidence degree calculating means 118 is a means for obtaining the integrated coincidence degree from the coarse structure coincidence degree Sa and the fine structure coincidence degree Sm as described below, instead of the determination processing in the person determining means 117 described above.

This integrated coincidence degree calculating means 118 is not essential in the person collation device of the present invention, and will be used when another embodiment is adopted. Further, in this case, the person determination means 117 does not perform threshold processing on the coarse structure coincidence degree Sa and the fine structure coincidence degree Sm independently as described above, but performs threshold processing on the obtained integrated coincidence degree. It is determined which registered person image 1212 the person in the input person image is a suspicious person.

The integrated coincidence degree calculating means 118 determines the weight w by referring to the read registered person image quality 1215. If the registered person image quality 1215 is higher than the image quality threshold “level 7” and high in quality, a large w is set, otherwise a small w is set.

Alternatively, a larger w may be determined as the registered person image quality 1215 is higher. In this case, the weight value for the registered person image quality 1215 may be determined as a simple straight line shape (proportional relationship) in the case of graphing, but as in a logarithmic function, an upward convex curve shape or an exponential function is used. You may determine as a downward convex curve shape. The maximum value of the weight w is 0.5, for example.

Then, the integrated coincidence degree calculating means 118 obtains the integrated coincidence degree S by the following equation using the weight w.

(Formula 1) S=(1−w)×Sa+w×Sm

That is, the higher the image quality of the registered person image is, the higher the contribution of the fine structure matching degree is to the integrated matching degree.

Then, the person determination unit 117 performs threshold processing on the integrated coincidence degree S and determines that the person of the registered person image is included in the input person image. The threshold value can be set to 80% of the maximum value that the integrated coincidence can take by previously knowing the maximum value through experiments.

Alternatively, the weight w may be set according to the image quality of the input person image. In this case, the value of the integrated coincidence degree can be calculated by directly reflecting the image quality of the actually photographed input person image.

<Operation of image monitoring device>
Hereinafter, the operation of the image monitoring process by the image monitoring apparatus 20 to which the present invention is applied will be described with reference to the flowchart shown in FIG. The operation described below is executed each time one input image is acquired.

In step S100, the imaging device 2 captures an input image by capturing an image of the monitoring area 100 and inputs the input image to the image acquisition unit 10.

In step S110, the face detection unit 111 attempts to detect a face area from the input image by a well-known method. If no person is included in the input image and the face area cannot be detected (No branch of step S120), the process returns to step S100 to acquire the input image at the next time.

When the face area exists (Yes branch of step S120), the face detection unit 111 extracts the input human image from the inside of the detected face area (step S130).

In step S140, the detected size of the face image detected by the face detection unit 111 is checked to determine the image quality of the input person image and the levels are classified.

The image quality evaluation is not limited to the face detection size as described above, and the contrast may be determined by examining the distance between other characteristic parts, the average brightness and the brightness dispersion.

In step S150, the tracking unit 114 tracks the face images that can be determined to be the same person in the input image, collects them, and stores them in the storage unit 12 as series data.

In step S160, when the image quality of the input person obtained in step S140 does not exceed the threshold value and is of low quality, the collation result is not output from the input image of the person and additional accumulation is performed in the series data in the previous step. It will be stopped (No branch).

Although not shown in the figure, as described with reference to FIG. 6, when the input person image quality of the series data does not become high quality and the tracking ends, the highest input person image quality among them is obtained. It is assumed that the following steps S170 to S200 are performed using the input person image.

Even in that case, if the tracking continues unnaturally for a long time and the image quality of the input person does not become high, for example, a person who performs a so-called suspicious behavior, such as intentionally listening to the surroundings at a position far from the imaging device 2. Therefore, steps S170 to S200 may be performed using the input person image having the highest input person image quality up to that point, without waiting for the end of tracking.

Steps S170 to S200 are repeatedly executed for each registered person image to be matched when the input person image has high quality.

In step S170, the coarse structure matching unit 115 obtains the coarse structure feature amount from the input human image, the fine structure comparison unit 116 obtains the fine structure feature amount from the input human image, and stores the registered human image to be processed. The coarse structure feature amount 1213 and the fine structure feature amount 1214 are read from the registrant data 121 of the part 12, and the coarse structure coincidence degree and the fine structure coincidence degree representing the degree of similarity are obtained.

When the registered person image quality 1215 of the registered person image to be processed is of high quality (Yes branch of step S180), the person determination unit 117 determines that the coarse structure matching degree is equal to or higher than the first threshold value and is fine. When the structural matching degree is equal to or higher than the second threshold value, the suspicious individual detection is determined (steps S190, S210, S220).

On the other hand, when the quality is low (No branch of step S180), the person determination means 117 determines the suspicious person when the rough structure matching degree is equal to or higher than the third threshold value larger than the first threshold value. (Steps S200, S210, S220).

The operation shown in FIG. 7B may be performed instead of step S190. This is a case where the integrated coincidence degree calculation unit 118 obtains the integrated coincidence degree.

In step S300, the integrated coincidence degree calculating means 118 determines the weight w used to obtain the integrated coincidence degree from (Equation 1) from the coarse structure coincidence degree and the fine structure coincidence degree.

Then, in step S310, the integrated coincidence degree calculating unit 118 obtains the integrated coincidence degree by (Equation 1), and in step S210, the obtained integrated coincidence degree is subjected to threshold processing to determine a suspicious individual. If is less than the threshold value, the process returns to step S100. On the other hand, if the degree of integration coincidence is equal to or more than the threshold value, a signal for detecting a suspicious person in the registered person image is output to the notification device 3, and the process returns to step S100 (step S220).

As described above, in the image monitoring apparatus according to the embodiment of the present invention, it is rough that both the input person image quality and the registered person image quality indicating the degree of the quality of the registered person image are high quality equal to or higher than the image quality threshold value. By determining whether or not the person of the registered person image is photographed in the input person image using the degree of structure agreement and the degree of fine structure agreement, it is possible to perform person determination with high accuracy.

Further, the image quality of each registered person image and the input person image is evaluated, and referred to at the time of matching, and the integrated matching degree using the coarse structure matching degree and the fine structure matching degree is determined according to the level of the image quality. By making the calculation method different, it is possible to suppress the acceptance of others and the rejection of the person and to accurately determine the person. As a result, even if the image quality of the registered person image is low, the person can be accurately determined. If the registered person image is of low quality, the person determination is performed using only the coarse structure matching degree without using the fine structure matching degree. Therefore, since there is little information on which the person determination is based, it is possible to suppress the acceptance of others by using a strict threshold as the threshold used for the person determination.

Further, when obtaining the integrated coincidence degree, the higher the quality of each of the registered person image and the input person image, the more effectively the information of the fine structure coincidence degree can be used, so that the acceptance of others can be suppressed.

In addition, assuming that the input person image representing the person tracked by the input person image becomes high quality at some point, the person determination using the input person image that is not high quality is not performed. If the high-quality input person image is not obtained, the person determination is not simply performed, but if the person is a target suspicious person, it is overlooked. Therefore, by performing the person determination using the input person image having the highest image quality in the series of data, it is possible to obtain a person determination result that is as trustworthy as possible.

<Modification>
Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments. For example, in the present embodiment, an example has been described in which face matching is performed using a rough structure feature amount regarding a face part as a rough face structure and a fine structure feature amount regarding a structure finer than the face part. However, the present invention is not limited to this. It is not something that will be done.

For example, from each of the input person image and the registered person image, a whole-body image representing the whole body of the person is detected, and a coarse structure feature amount related to a coarse structure of the whole body and a fine structure feature amount related to a finer structure thereof are extracted, Person verification may be performed. Specifically, as a rough structure of the whole body, the rough shape, body type (obesity type, slimming type, sportsman type, etc.), height (small size, middle height, large size), color of clothes and combination of upper and lower clothes, etc. In addition to extracting the structural features, the facial features (eye nose and mouth shape, face outline (round face, face length)), and the fine features related to the pattern of clothes and neckties as finer structures are also extracted, You may collate.

As described above, those skilled in the art can make various modifications according to the embodiment within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Face collation device 2 Imaging device 3 Notification device 10 Image acquisition unit 11 Image processing unit 12 Storage unit 13 Output unit 20 Image monitoring device 111 Face detection means 112 Feature point extraction means 113 Quality evaluation means 114 Tracking means 115 Rough structure matching means 116 Fine structure collating means 117 Person judging means 118 Integrated coincidence degree calculating means

Claims (5)

An image acquisition unit that acquires a captured image of a predetermined area,
Detecting means for detecting an input person image from the photographed image;
A coarse structure feature amount representing image features relating to a coarse structure forming a person, and a fine structure feature amount representing image features relating to a fine fine structure in the person, which are extracted from a registered person image of a target person. Storage means for storing the coarse structure feature amount and the fine structure feature amount for each of the target persons,
A quality evaluation means for evaluating the image quality of the input person, which represents the degree of quality of the input person image;
A rough structure matching unit that extracts the rough structure feature amount from the input person image and calculates a rough structure matching degree with the rough structure feature amount stored in the storage unit;
A fine structure matching unit that extracts the fine structure feature amount from the input human image and calculates a fine structure matching degree with the fine structure feature amount stored in the storage unit;
A person determination unit that determines whether or not the target person is photographed in the input person image using at least the rough structure matching degree,
Equipped with
The person determination means,
If both the input person image quality and the registered person image quality indicating the degree of quality of the registered person image are high quality equal to or higher than the image quality threshold value, the input person image is calculated using the coarse structure coincidence degree and the fine structure coincidence degree. A person collation apparatus for determining whether or not the target person is photographed. The person determination means,
When both of the input person image quality and the registered person image quality are the high quality, the input person when the coarse structure matching degree is a first threshold value or more and the fine structure matching degree is a second threshold value or more It is determined that the target person is photographed in the image,
If the input person image quality is the high quality but the registered person image is of a low quality less than the image quality threshold value, the input is performed when the coarse structure matching degree is equal to or higher than a third threshold value higher than the first threshold value. The person matching apparatus according to claim 1, wherein the person matching unit determines that the target person is photographed in the person image. The person determination means,
When both the input person image quality and the registered person image quality are the high quality, an integrated coincidence degree obtained by integrating the coarse structure coincidence degree and the fine structure coincidence degree is calculated, and the input person image is calculated from the integrated coincidence degree. Determine whether the target person is photographed in the image,
If the image quality of the input person is the high quality, but the registered person image is of a low quality less than the image quality threshold value, it is determined from the rough structure matching degree whether or not the target person is photographed in the input person image. The person verification apparatus according to claim 1, wherein The person determination means,
When both the input person image quality and the registered person image quality are the high quality, the higher the registered person image quality, the higher the contribution of the fine structure coincidence degree to the integrated coincidence degree is to calculate the integrated coincidence degree. The person verification apparatus according to claim 3, wherein the person verification apparatus is a person verification apparatus. The person verification device further includes
Tracking means for tracking the input person images detected from the plurality of captured images sequentially acquired for the same person, and generating series data in which the traceable input person images are associated with each other,
The person determination means,
For each of the series data, if the highest input person image quality of the input person image quality of the input person image included in the series data is low quality that is less than the image quality threshold, the coarse structure matching degree is the first. The person collation apparatus according to claim 2 , wherein when the number is equal to or more than a third threshold value higher than the threshold value, it is determined that the target person is captured in the input person image included in the series data.