JP7214830B2 - Image matching system - Google Patents

Description

The present invention relates to an image collation system for collating an image captured by an imaging device with an image registered in a database.

2. Description of the Related Art In recent years, facilities such as airports, train stations, shopping malls, and the like, which are visited by an unspecified number of people, are equipped with video monitoring systems for security purposes such as deterring crimes and preventing accidents. Examples of such image monitoring systems include an abandoned object detection system that monitors an image to detect an abandoned object, and a person search system that searches for a designated person in an image. In addition, there are cases where a video analysis system is constructed for marketing purposes, not limited to video monitoring systems for security purposes. Such video analysis systems include, for example, a face image analysis system that identifies persons in a facility and identifies attributes such as age and sex.

WO2016/151802

In the case of a video surveillance system that handles facial images and aims to identify a person or identify attributes, it is necessary to register in advance the facial image data of a person to be identified. For example, Patent Literature 1 discloses a face collation system that extracts feature amounts from photographed image data and performs matching by comparing them with feature amounts registered in a database.

In conventional face image analysis systems, especially systems aimed at matching such as face matching, it is difficult to achieve stable matching accuracy regardless of environmental changes such as installation conditions of imaging devices such as cameras and sunlight conditions. Met. For example, in a video surveillance system in which face images taken in an indoor environment with good lighting conditions are pre-registered, the environment of the camera that takes pictures in real time changes every moment, and facial images may not be visible at night or in the direction of the lighting. If a shadow is formed, the image will look completely different from the pre-registered face image, so the matching accuracy tends to decrease.

In order to improve these, it is important to make the video quality of the camera video taken in real time and the pre-registered face image the same level as much as possible. Conventionally, the quality of pre-registered face images is recorded, and the camera settings are adjusted in real time to bring the quality of the camera video closer to the quality of the pre-registered face images. It was done.

General cameras are equipped with correction functions such as auto white balance, auto gain, and wide dynamic range (WDR) to reduce the effects of environmental conditions such as lighting and maintain a certain level of image quality. designed to achieve a certain level of image quality. However, if the camera setting is made close to the quality of the pre-registered face image, the camera image can only be captured for the purpose of improving the accuracy of face matching. Therefore, in some cases, the video quality may be insufficient for monitoring purposes. In other words, in the case of a video surveillance system, it becomes difficult to satisfy both the surveillance purpose and the verification purpose.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image matching system capable of obtaining stable matching accuracy without changing the settings of the imaging device.

In order to achieve the above object, in the present invention, an image matching system is configured as follows.
That is, in an image matching system for matching a captured image captured by an imaging device with a registered image registered in advance, storage means for storing the registered image and its quality, and specifying means for specifying the quality of the captured image. adjusting means for adjusting the quality of said registered image so as to approach the quality of said photographed image based on the quality of said photographed image and the quality of said registered image; and said registered image adjusted by said adjusting means. and a collating means for collating the photographed images using a collating means.

As described above, the image collation system according to the present invention is configured to temporarily adjust the registered image so as to approach the quality of the photographed image and use it for collation of the photographed image. Accordingly, the photographed images can be collated while the quality of the photographed image and the quality of the registered image are adjusted to a level suitable for image collation (for example, approximately the same level). Therefore, since the quality of the photographed image does not need to be adjusted, stable matching accuracy can be obtained without changing the settings on the imaging device side. Furthermore, since there is no need to change the settings on the imaging device side, the captured image can be used not only for verification purposes but also for monitoring purposes.

Note that the adjustment means adjusts the quality of the captured image when the difference between the quality of the captured image and the quality of the registered image is equal to or greater than a predetermined value, when the quality of the captured image deviates from a predetermined condition, or when a pre-scheduled timing has arrived. In this case, it is preferable to adjust the quality of the registered image. As a result, it is possible to prevent the adjustment of the quality of the registered image from being performed even when it is not necessary, and the processing load can be reduced.

Here, in the image matching system according to the present invention, various qualities that may affect matching accuracy can be used as the quality. 1 or more can be used.

Further, the image matching system according to the present invention can be used in various systems. It is possible to provide a face matching system that performs matching of the detected face image using the registered image whose quality is adjusted so that the quality approaches that of the detected face image.

According to the present invention, it is possible to provide an image matching system capable of obtaining stable matching accuracy without changing the settings of the imaging device.

1 is a diagram showing a schematic configuration of a face matching system according to one embodiment of the present invention; FIG. 2 is a diagram showing a configuration example of an image processing device in the face matching system of FIG. 1; FIG. 2 is a diagram showing an example of a flow chart of face matching by the face matching system of FIG. 1; FIG.

An embodiment of the present invention will be described with reference to the drawings. A face matching system, which is an example of an image matching system, will be described below as an example.
FIG. 1 shows a schematic configuration of a face matching system according to one embodiment of the present invention. The face matching system of this example includes an imaging device 10 , an image processing device 20 , a recording device 30 and an operation terminal 40 .

The imaging device 10 is basically set to be suitable for use for monitoring purposes, and the settings are not changed one by one for the purpose of verification. The imaging apparatus 10 also has a function of automatically adjusting the quality of the captured image in accordance with changes in environmental conditions (lighting/unlit, weather changes, etc.) at the time of shooting. In FIG. 1, three cameras, ie, a network camera 10-1, an analog camera 10-2, and a network camera 10-3 are illustrated as the imaging device 10, but the number of these cameras is arbitrary. The network cameras 10-1 and 10-3 incorporate encoders for encoding captured images for communication and are directly connected to the network. The analog camera 10-2 is connected to the network via a separate encoder 12 that encodes captured images for communication. An image captured by the imaging device 10 is transmitted to the image processing device 20 through the network.

The recording device 30 causes a hard disk or the like to store a face image of a person registered in advance and data of its quality. Note that the recording device 30 may also store data such as attributes such as a person's name and gender, and facial feature amounts obtained by analyzing a facial image. As a face image of a person to be registered in advance, for example, one photographed in an indoor environment with good lighting conditions is used. Contrast, white balance, compression level, brightness level, etc. are used as face image quality, but various other qualities that may affect the accuracy of face matching may also be used.

The image processing device 20 detects a face image of a person from an image captured by the imaging device 10 and performs face matching to match the detected face image with face images pre-registered in the recording device 30 . Hereinafter, when distinguishing between face images detected from captured images and pre-registered face images, the former will be referred to as "detected face images" and the latter will be referred to as "registered face images". Details of the image processing device 20 will be described later.
The operation terminal 40 is a terminal used by a user of the face matching system, and displays the result of face matching by the image processing device 20, the captured image by the imaging device 10, and the like.

FIG. 2 shows a configuration example of the image processing device 20 in the face matching system of this example.
The image processing device 20 includes a captured image receiving unit 21, an image quality analyzing unit 22, a face detecting unit 23, a registered data acquiring unit 24, a quality evaluating unit 25, an image quality adjusting unit 26, and a matching unit 27. Prepare.

The captured image receiving unit 21 communicates with the imaging device 10 and receives captured images obtained by imaging with the imaging device 10 in real time.
The image quality analysis unit 22 analyzes the captured image received by the captured image receiving unit 21 and specifies multiple types of quality such as contrast, white balance, compression level, and brightness level of the captured image. For example, for white balance, the RGB levels (the levels of each color of R (Red), G (Green), and B (Blue)) of each pixel included in the captured image are analyzed. The registered face image is corrected so as to approach the RGB level. As for the luminance level, a certain part (for example, the nose) is analyzed from the position of the face detected from the photographed image. The registered face image is corrected so as to approach this luminance level. It should be noted that when the imaging device 10 associates the quality data with the captured image and transmits it, the quality of the captured image can be obtained without analyzing the captured image. The captured image quality identified by the image quality analyzer 22 is temporarily stored in memory along with the captured image for subsequent processing.

The face detection unit 23 performs face detection processing for detecting a face image included in the captured image received by the captured image receiving unit 21 . Various known techniques can be used for face detection processing.
The registered data acquisition unit 24 communicates with the recording device 30 to acquire the registered face image recorded in the recording device 30 and its quality data.

When the face detection unit 23 detects a face image from the captured image, the quality evaluation unit 25 evaluates the quality of the captured image specified by the image quality analysis unit 22 and the registered face image obtained by the registration data obtaining unit 23. and the quality of the registered face image, and it is checked whether the difference is equal to or greater than a predetermined value, and the necessity of adjusting the quality of the registered face image is evaluated. As an example, the difference between the quality value of the photographed image and the quality value of the registered face image is calculated for each quality type such as contrast, white balance, compression level, and brightness level, and the threshold value set in advance for each quality type is calculated. If the difference is found to be equal to or greater than the threshold, it is determined that the quality of the registered face image needs to be adjusted. As another example, the difference between the quality value of the captured image and the quality value of the registered face image is calculated for each quality type, multiplied by a weighting factor preset for each quality type, summed, and the total value is calculated in advance. It is compared with a set threshold value, and if the total value is equal to or greater than the threshold value, it is determined that adjustment of the quality of the registered face image is necessary. Note that these are merely examples, and other methods may be used to compare the quality of the captured image and the quality of the registered face image.

Here, when the quality of the registered face images recorded in the recording device 30 is uniform (for example, when all the registered face images are captured under the same shooting conditions), the quality evaluation unit 25 may be selected as a representative registered face image and compared with the quality of the photographed image. In addition, when a plurality of registered face images can be grouped with those having similar quality, the quality evaluation unit 25 selects a representative registered face image for each group and compares it with the quality of the captured image. good too. On the other hand, if the quality of the registered face images is not uniform or cannot be grouped, the quality evaluation unit 25 compares each of the plurality of registered face images recorded in the recording device 30 with the quality of the captured image. There is a need to do.

The image quality adjustment unit 26 adjusts the quality of the registered face image acquired from the recording device 30 when the quality evaluation unit 24 determines that the quality of the registered face image needs to be adjusted. The quality of the registered face image is adjusted so that the quality of the registered face image approaches the quality of the captured image. It is generally determined that the quality of the registered face image needs to be adjusted when the quality of the captured image deteriorates due to deterioration of the shooting environment. will be applied. The adjustment of the quality of the registered face image may be performed for all types of quality, but the quality of the type whose difference from the quality value of the captured image is above the threshold It is possible to reduce the processing load by limiting the processing to the quality of the type.

When the face detection unit 23 detects a face image of a person from the photographed image, the collation unit 27 performs face collation for collating the detected face image with the registered face image. The collation unit 27 normally collates the registered face image itself acquired from the recording device 30, but for the registered face image determined to require quality adjustment, the registered face image after adjustment by the image quality adjustment unit 26. match with A collation result by the collating unit 27 is transmitted to the operation terminal 40 and displayed on the screen of the operation terminal 40 .

FIG. 3 is a diagram showing an example of a flow chart of face matching by the face matching system of this example.
As pre-processing, a face image to be registered (registered face image) and its quality data are recorded in the recording device 30 (steps S11 and S12), and it is determined whether or not the registration is completed (step S13). If unprocessed registered face images remain, it is determined that the registration is not completed, and steps S11 and S12 are repeated until there are no more unprocessed registered face images. When there are no more unprocessed registered face images, it is determined that the registration is finished, and the preprocessing is finished.

After finishing the pre-processing, the captured image receiving unit 21 acquires the captured image from the imaging device 10 (step S14). Next, the image quality analysis unit 22 analyzes the captured image acquired by the captured image receiving unit 21 and specifies its quality (step S15). Next, the face detection unit 23 performs face detection processing on the captured image acquired by the captured image receiving unit 21 (step S16). Next, it is determined whether or not a face image has been detected from the captured image (step S17). If the face image is not detected from the captured image, the process returns to step S14 to acquire the next captured image from the imaging device 10 . When a face image is detected from the captured image, the quality evaluation unit 25 evaluates the necessity of adjusting the quality of the registered face image (step S18). In this example, the quality of the registered face images is sequentially read out from the recording device 30 by the registered data acquiring unit 24, compared with the quality of the photographed image, and the quality of the registered face images having a difference in quality equal to or greater than a predetermined value is adjusted. determine that it is necessary. When it is determined that the quality of the registered face image needs to be adjusted, the image quality adjustment unit 26 adjusts the quality of the registered face image in a direction to bring the quality of the registered face image closer to the quality of the captured image (that is, the quality of the detected face image). is adjusted (step S19). Next, the matching unit 27 performs matching of the detected face image against the registered face image acquired from the recording device 30 or the registered face image after adjustment by the image quality adjustment unit 26 (step S20).

Specifically, the image processing device 20 of this example analyzes the captured image obtained from the imaging device 10 in real time and records the quality such as contrast, white balance, compression level, and luminance level. Then, when there is a scene in which a face image can be obtained from the photographed image, the quality data specified by analyzing the photographed image is compared with the quality data of the registered face image recorded in the recording device 30. , to clarify the difference in quality between the registered face image and the photographed image by comparing the image quality. Based on the result of this comparison, a registered face image having a quality difference greater than or equal to a predetermined value is searched, and brightness level correction, contrast correction, etc. are performed on the registered face image so as to bring it closer to the real-time photographed image. Apply. In addition, the compression level varies depending on the imaging device 10, and there is a tendency to apply a high compression level to real-time captured images in particular. approach the level. In this way, when performing face matching, the pre-registered registered face image is temporarily corrected, and the detected face image detected from the captured image is compared, thereby making both image quality levels the same. performs face matching processing.

As described above, in the face matching system of this example, the recording device 30 stores registered face images registered in advance and their quality data. In the image processing device 20, the image quality analysis unit 22 identifies the quality of the captured image captured by the imaging device 10, the face detection unit 23 detects a face image from the captured image, and the quality evaluation unit 25 When the face detection unit 23 detects a face image from the photographed image, the necessity of adjusting the quality of the registered face image is evaluated, and the image quality adjustment unit 26 determines the quality of the registered face image by the quality evaluation unit 25. When it is determined that adjustment is necessary, based on the quality of the captured image and the quality of the registered face image, the quality of the registered face image is adjusted so as to approach the quality of the captured image. Using the registered face image whose quality has been adjusted by the adjustment unit 26, the detected face image is collated.

As described above, in the face matching system of this example, the registered face image is temporarily adjusted so as to approach the quality of the captured image (that is, the quality of the detected face image), and is used for matching of the detected face image. I have As a result, the quality of the detected face image and the quality of the registered face image can be adjusted to a level suitable for image matching (for example, approximately the same level), and matching of the detected face image can be performed. Therefore, since the quality of the photographed image does not need to be adjusted, stable matching accuracy can be obtained without changing the settings on the imaging device side. Furthermore, since there is no need to change the settings on the imaging device side, the captured image can be used not only for verification purposes but also for monitoring purposes.

Here, in the face matching system of this example, the storage means according to the present invention is constituted by the recording device 30, the specifying means according to the present invention is constituted by the image quality analysis section 22, and the adjusting means according to the present invention is the image quality Although the adjusting section 26 is configured and the collating means according to the present invention is configured by the collating section 27, it goes without saying that other configurations may be used.

In the above description, the quality evaluation unit 25 evaluates the necessity of adjusting the quality of the registered face image by comparing the quality of the captured image and the quality of the registered face image. may evaluate the need for adjusting the quality of the registered face image. For example, the quality of the captured image may be compared with a predetermined condition (a condition that does not depend on the quality of the registered face image) to evaluate the necessity of adjusting the quality of the registered face image. That is, as an example, a reference range is set in advance for each quality type, and if the value of the quality of the captured image deviates from the reference range, it is determined that the quality of the registered face image needs to be adjusted. good.

Further, for example, when a pre-scheduled timing arrives, it may be determined that the quality of the registered face image needs to be adjusted. As an example, when the timing of turning on/off the lighting equipment around the imaging device 10 is scheduled in advance, it may be switched whether to adjust the quality of the registered face image according to the timing. This method is effective when the timing of environmental changes that affect the quality of captured images is known in advance.

As described above, the case where the image matching system according to the present invention is applied to a face matching system has been described as an example, but the present invention is not limited to such a face matching system. Needless to say, it can be widely applied to various image matching systems for matching other images with registered images.
Further, the present invention provides, for example, a method and system for executing the processing according to the present invention, a program for realizing such a method and system using hardware resources such as a memory and a processor, and a memory for storing the program. It is also possible to provide it as a medium or the like.

INDUSTRIAL APPLICABILITY The present invention can be used in various image collation systems for collating a captured image captured by an imaging device with a registered image registered in advance.

10: imaging device, 20: image processing device, 30: recording device, 40: operation terminal,
21: Captured image receiving unit 22: Image quality analysis unit 23: Face detection unit 24: Registered data acquisition unit 25: Quality evaluation unit 26: Image quality adjustment unit 27: Verification unit

Claims (4)

In an image matching system for matching a captured image captured by an imaging device with a pre-registered registered image,
a storage means for storing the registered image and its quality;
identifying means for identifying the quality of the captured image;
an adjustment unit that compares the quality of the captured image with the quality of the registered image and adjusts the quality of the registered image so that the quality of the registered image approaches the quality of the captured image;
a matching means for matching the photographed image using the registered image after adjustment by the adjusting means;
When a plurality of registered images registered in advance for a plurality of matching objects and having similar quality are grouped, the adjustment means selects a representative registered image for each group. is selected and compared with the quality of the photographed image. In the image matching system according to claim 1,
The adjustment means adjusts the quality of the captured image when the difference between the quality of the captured image and the quality of the registered image is equal to or greater than a predetermined value, when the quality of the captured image deviates from a predetermined condition, or when a pre-scheduled timing has arrived. , an image matching system characterized in that quality adjustment is applied to said registered image. In the image matching system according to claim 1 or claim 2,
The image matching system, wherein the quality is one or more of contrast, white balance, compression level, and luminance level. In the image matching system according to any one of claims 1 to 3,
The registered image is a face image of a person,
characterized in that when a face image of a person is detected from the photographed image, the detected face image is compared using the registered image whose quality is adjusted so that the quality of the detected face image approaches that of the detected face image. image matching system.

Images