JP2018107665A - Imaging apparatus, dirt detection system, and dirt detection method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of detecting temporal dirt in an imaging part, a dirt detection system, and, a dirt detection system.SOLUTION: The present invention relates to an imaging apparatus capable of detecting temporal dirt in an imaging part. The imaging apparatus comprises: an imaging part; a matching degree calculation part which compares a partial reference image of a reference image that is picked up by the imaging part with a partial input image, corresponding to the partial reference image, in each of multiple input images that are picked up by the imaging part and calculates a matching degree of each of the partial input images to the partial reference image; a maximum value detection part for calculating, for each of multiple periods in which start times are preceding to and following each other, a maximum value within the multiple matching degrees regarding the multiple partial input images of the multiple input images that are picked up during the relevant period; and a dirt discrimination part for discriminating whether temporal dirt is generated in the imaging part based on temporal changes of the multiple maximum values that are calculated for the multiple periods.SELECTED DRAWING: Figure 1

Description

The present invention relates to an imaging apparatus, a dirt detection system, and a dirt detection method. More specifically, the present invention relates to an imaging apparatus, a dirt detection system, and a dirt detection method suitable for detecting dirt over time of an optical lens and a cover of an imaging unit of a face authentication apparatus.

2. Description of the Related Art Conventionally, there has been known a face authentication device that performs personal authentication by imaging an authentication target person's face using an imaging device and comparing the captured face image with a pre-registered face image.

As another technique for processing such a captured image, for example, regarding a monitoring system using a monitoring camera, the input image and the reference image are divided into a plurality of blocks and compared, and monitoring is performed when the degree of mismatch is large. A surveillance camera tampering detection apparatus that determines that a tampering action is being performed on a camera is disclosed (for example, see Patent Document 1). In the apparatus described in Patent Document 1, the reference image is updated based on the time and the brightness of the entire image in order to cope with the variation in illumination.

In addition, regarding image identification on the inspection line, such as monitoring of irregularities in a bookbinding device, the correlation value of the captured image with respect to the sample image is obtained by comparing the feature amount of the captured image by the imaging means with the feature amount of the sample image stored in advance An image processing apparatus that identifies an image is disclosed (for example, see Patent Document 2). In the apparatus described in Patent Document 2, a correlation value that is equal to or greater than a determination threshold among correlation values calculated each time each image is identified is stored as time-series data, and the correlation value is calculated from the correlation between a plurality of correlation values. Extract trends over time. Specifically, an inclination value is calculated from a plurality of correlation values, and a warning is output when the inclination value is a negative value continuously for a predetermined number of times. Causes of the temporal change tendency include dirt and clouding of the optical lens of the image pickup means, a decrease in illumination illuminance, and a change in the state of the imaged image with time (such as a decrease in print density of the printed product).

Further, regarding a technique for obtaining a time required for the vehicle to travel between predetermined points, the image is obtained by imaging with an imaging unit when light is emitted from a light emitting unit to a predetermined imaging region where an imaging target does not exist. Brightness calculating means for calculating the brightness of a predetermined pixel area of the captured image, and determination means for determining whether the brightness is smaller than a threshold value according to an imaging condition and a light emission condition at the time of capturing the captured image; An image processing device for road traffic is disclosed that includes notifying means for notifying a decrease in the light emission intensity of the light emitting unit when the determining means determines that the light intensity is small (see, for example, Patent Document 3).

JP 2005-252479 A JP-A-11-110547 JP 2009-181544 A

However, in the conventional face authentication device, if the optical member such as the optical lens of the image pickup unit or the cover of the optical lens becomes dirty, the authentication accuracy may be lowered even if the appearance of the input image input from the image pickup unit is not changed. there were. Such dirt includes sudden dirt such as adhering matter, and time-dependent dirt that gradually gets dirty every day due to the passage of time, and it is possible to express the time-dependent dirt clearly by numerical values. There wasn't.

In the apparatus described in Patent Document 1, since the threshold value processing is performed on the inconsistency of the image while updating the reference image, a short-term change may be detected, but a long-term change such as a time-dependent stain that changes little by little is detected. It cannot be detected.

Moreover, in the apparatus described in Patent Document 1, the minimum value of the mismatch degree for each block is finally added by the adding unit, and the result is subjected to threshold processing to determine the presence or absence of the disturbing action. That is, since the entire input image is finally compared with the reference image, if the entire input image is not similar to the entire reference image, it is determined that there is a disturbing action. For this reason, both the reference image and the input image need to be images obtained by capturing only the background that does not change. However, when shooting a place where there is a lot of traffic or where the background changes greatly, or when shooting a place where the lighting environment changes due to the time, such as outdoors or near the window, it is possible to compare. It is difficult to obtain an input image and a reference image that can be obtained.

Since the apparatus described in Patent Document 2 uses time-series data of correlation values that have been determined to be OK at or above the determination threshold, the time-series data includes the imaged image object, its background, dirt on the optical lens of the imaging means, and the like. Including multiple factors. Therefore, it is difficult to detect the contamination over time of the optical lens of the image pickup means separately from other factors. Further, as in face authentication, when the change in the captured image object itself for which the degree of coincidence is obtained is large, if the time series data includes factors other than the contamination of the optical lens over time, Dirt cannot be detected accurately.

Further, the contamination of the optical lens over time affects the entire captured image and lowers the luminance of each pixel. Therefore, the apparatus described in Patent Document 3 is detected as a decrease in emission intensity. Therefore, even with this technique, it is impossible to detect the contamination of the optical lens over time from the change in the characteristics of the hardware.

The present invention has been made in view of the above-described present situation, and an object thereof is to provide an imaging apparatus, a dirt detection system, and a dirt detection method capable of detecting a stain with time of an imaging unit. It is.

The present invention is an image pickup apparatus capable of detecting dirt over time of an image pickup unit, the image pickup unit, a partial reference image of a reference image shot by the image pickup unit, and a plurality of inputs shot by the image pickup unit A plurality of periods in which the start time is mutually compared, and a coincidence calculation unit that compares the partial input images corresponding to the partial reference images of each of the images and calculates the coincidence of each partial input image with respect to the partial reference image A maximum value detecting unit for obtaining a maximum value of a plurality of matching degrees for a plurality of partial input images of a plurality of input images photographed within the period, and a plurality of values obtained for the plurality of periods. And a dirt determination unit that determines whether or not the image pickup unit is contaminated with time based on a change with time of the maximum value.

Further, according to the present invention, in the above invention, the coincidence degree calculation unit normalizes each of the partial reference image and the plurality of partial input images with brightness information included in the partial image, and normalizes the partial reference The degree of coincidence is calculated based on an image and a plurality of normalized partial input images.

Also, the present invention is characterized in that, in the above invention, the coincidence calculation unit calculates the coincidence based on contour information included in each of the partial reference image and the plurality of partial input images. .

Further, the present invention is characterized in that, in the above invention, the coincidence calculation unit divides the reference image and the input image into a plurality of partial areas corresponding to each other, and calculates the coincidence for each partial area. And

Further, the present invention is the above invention, wherein the maximum value detection unit calculates all the matches calculated by the match degree calculation unit by dividing the plurality of input images taken within the period into a plurality of partial regions, respectively. The maximum value is obtained from the degree.

Further, in the present invention according to the above invention, the maximum value detection unit may calculate a plurality of matches calculated by the match level calculation unit by dividing the plurality of input images taken during the period into a plurality of partial regions, respectively. The maximum value is obtained for each of the corresponding partial areas based on the degree, and the stain determination unit determines the time-dependent change based on the change over time of the plurality of maximum values for the partial areas obtained for the plurality of periods. It is characterized in that whether or not dirt is generated is determined for each partial area.

In the above invention, the present invention further includes a comparison extraction unit that compares the reference image and each of the plurality of input images to extract a region with a small change, and the matching degree calculation unit The degree of coincidence is calculated in a small area.

Further, according to the present invention, in the above invention, the coincidence calculation unit compares the partial reference images of each of the plurality of reference images captured by the imaging unit with the partial input images, and calculates the coincidence. It is characterized by calculating.

Further, the present invention is the above invention, wherein the maximum value detection unit obtains the maximum value for each of the plurality of periods, and the stain determination unit performs the process for each of the plurality of periods. It is characterized by determining the presence or absence of the occurrence of dirt over time.

Further, the present invention is characterized in that, in the above invention, the stain determination unit determines whether or not the stain with time has occurred by comparing the maximum value with a predetermined threshold value.

In addition, the present invention is a dirt detection system capable of detecting dirt over time of an imaging unit, and includes a partial reference image of a reference image taken by an imaging unit, and a plurality of input images taken by the imaging unit. A degree-of-matching calculation unit that compares the degree of matching of each partial input image with the partial reference image by comparing with the partial input image corresponding to each of the partial reference images, and each of a plurality of periods before and after the start time A maximum value detecting unit for obtaining a maximum value of a plurality of matching degrees for a plurality of partial input images of a plurality of input images photographed within the period, and a plurality of maximum values obtained for the plurality of periods And a stain determination unit that determines whether or not the image pickup unit is contaminated with time based on a change with time.

In addition, the present invention is a dirt detection method for detecting dirt over time of an imaging unit, each of a partial reference image of a reference image taken by an imaging unit and a plurality of input images taken by the imaging unit Each of the partial input images corresponding to the partial reference image and calculating the degree of coincidence of each partial input image with respect to the partial reference image, and each of a plurality of periods before and after the start of each other A maximum value detecting step for obtaining a maximum value among a plurality of matching degrees for a plurality of partial input images of a plurality of input images photographed within the period, and a plurality of maximum values obtained for the plurality of periods. And a dirt determination step for determining whether or not dirt with time has occurred in the imaging unit based on a change with time.

According to the imaging device, the dirt detection system, and the dirt detection method of the present invention, it is possible to detect the dirt of the imaging unit over time.

It is a schematic diagram which shows the structure of the face authentication apparatus which concerns on Embodiment 1. FIG. FIG. 6 is a schematic diagram for explaining an example of a dirt detection method (image processing method) by the face authentication apparatus according to the first embodiment. 6 is a graph showing an example of a change with time of the maximum value of the degree of coincidence in the first embodiment. 5 is a flowchart illustrating an operation flow of the face authentication apparatus according to the first embodiment. FIG. 10 is a schematic diagram for explaining another example of a stain detection method (image processing method) by the face authentication apparatus according to the first embodiment. 10 is a schematic diagram for explaining an example of a dirt detection method (image processing method) by the face authentication apparatus according to the second embodiment. FIG. 10 is a graph showing an example of a change with time of the maximum value of the degree of coincidence for each partial region (block) in the second embodiment. FIG. 10 is a schematic diagram for explaining an example of a dirt detection method (image processing method) by the face authentication apparatus according to the third embodiment. It is a schematic diagram which shows the structure of the face authentication apparatus which concerns on Embodiment 4. FIG. 10 is a schematic diagram for explaining an example of a dirt detection method (image processing method) by the face authentication apparatus according to the fourth embodiment.

(Embodiment 1)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments of an imaging device, a dirt detection system, and a dirt detection method according to the invention will be described in detail with reference to the drawings. Here, an embodiment in which these are applied to the face authentication device will be described.

The face authentication apparatus according to the present embodiment is a face authentication apparatus that can detect a time-dependent stain on an imaging unit, particularly an optical member such as an optical lens or an optical lens cover. The term “dirt over time” refers to a stain that gradually accompanies every day due to the passage of time, and examples of the cause include dust, dust, and air pollutants.

The face authentication apparatus according to the present embodiment also performs personal authentication by capturing the face of the person to be authenticated using the image capturing unit and comparing the captured face image with a pre-registered face image. However, since a series of processes related to face authentication is general, a description thereof will be omitted, and a method for detecting dirt over time of the imaging unit will be described in detail below.

As shown in FIG. 1, the face authentication device according to the present embodiment includes an imaging unit 10 such as a CCD (Charge Coupled Device) camera, a control unit 20 that performs various controls and processes, a reference image to be compared, The storage unit 30 stores a plurality of calculated degrees of coincidence and their maximum values, various threshold values, various software programs, and the like. The control unit 20 includes a coincidence degree calculation unit 21, a maximum value detection unit 22, and a stain determination unit 23.

The degree of coincidence calculation unit 21 performs image collation, and partially includes a reference image previously captured by the imaging unit 10 and each of a plurality of input images sequentially captured by the imaging unit 10 after the reference image is captured. Compared with each region (hereinafter also referred to as a block), the degree of coincidence of each input image with respect to the reference image is calculated for each block.

More specifically, as shown in FIG. 2, the coincidence degree calculation unit 21 divides the reference image and each of the input images 1, 2,..., L, L + 1,. Divide into N, for example, 3 × 3 blocks (partial regions). Then, the partial reference image 41 corresponding to each block is compared with each partial input image 42 corresponding to the partial reference image 41, and the degree of coincidence of each partial input image 42 with respect to each partial reference image 41 is calculated. In this specification, the partial reference image and the partial input image mean images that constitute a part of the reference image and the input image, respectively. Further, the number of blocks and the division method (arrangement and shape) of the blocks are not particularly limited and can be set as appropriate.

In FIG. 2, when a person to be authenticated is photographed in some blocks (input image 1), when a light source such as lighting or the sun is reflected in some blocks (input image 2), passers-by in all blocks Is illustrated (input image L), a passerby is captured in some blocks (input image L + 1), and the degree of coincidence of each block in these cases is illustrated. Thus, the timing at which each input image is acquired is not particularly limited and can be set as appropriate, and may be either during authentication or during non-authentication.

The degree of coincidence is an index indicating the degree of similarity between images, and is represented by a value within a finite numerical range, for example, a value between −1 and +1, and is calculated using, for example, a normalized correlation coefficient. When the normalized correlation coefficient is used, the degree of coincidence is calculated based on the pixels in the entire region to be compared.

Each image data usually includes information related to brightness (brightness and darkness) and information related to a pattern, but according to the normalized correlation coefficient, each image data is normalized by information related to brightness. Will be. Therefore, the coincidence calculation unit 21 can calculate the coincidence based on the normalized partial reference image and partial input image, respectively. As a result, only the information related to the pattern is extracted from each image data, and the degree of coincidence is calculated based on the information related to the pattern, so the influence of the brightness change such as lighting on / off and illumination illumination change is canceled. Thus, the degree of coincidence can be calculated based only on the pattern of each image.

In addition to the method using the normalized correlation coefficient, the coincidence degree calculation unit 21 may calculate the coincidence degree by the following method (1) or (2), for example. The same effect as when used is obtained.
(1) Image collation (coincidence calculation) is performed after each image is normalized in advance with information related to brightness (brightness and darkness) included in the image data.
(2) Image collation (coincidence calculation) is performed based on information related to the contour not related to light and dark. For example, image collation is performed using an edge image obtained by differentiating image data.

The reference image can be appropriately photographed, but preferably includes only a subject other than the moving object, not including a moving object such as a person. However, when the optical member becomes dirty as a whole, it is only necessary that an object other than the moving object is shown in at least a part of the reference image.

For each of a plurality of periods (hereinafter also referred to as “maximum value detection periods”) in which the start times are before and after each other, the maximum value detection unit 22 performs a plurality of partial input images of a plurality of input images photographed within the period. Find the maximum value among multiple matches.

More specifically, as shown in FIG. 2, the maximum value detection unit 22 detects the L input images captured by the imaging unit 10 through a predetermined maximum value detection period every time. The degree of coincidence is compared for each block, and the maximum degree of coincidence (primary maximum value) is selected for each block. Further, the maximum value (secondary maximum value) is selected from the matching degrees (M × N primary maximum values) of all blocks, and stored in the storage unit 30. That is, for each maximum value detection period, the maximum one is extracted from all the matching degrees regarding all the partial input images of all the input images photographed throughout the period, and is stored in the storage unit 30 as time series data. As described above, instead of extracting the one with the highest degree of coincidence in each input image and using it as time-series data, the largest one is extracted from the degree of coincidence related to a plurality of input images within the maximum value detection period. By using the series data, it is possible to prevent the time series data from including the influence when the degree of coincidence temporarily decreases in the entire input image (for example, in the case of the input image L in FIG. 2). Therefore, it is possible to detect the contamination of the imaging unit 10 over time from other factors.

The length of the maximum value detection period is not particularly limited and can be set as appropriate, but is preferably at least one day, for example, one day. The number of input images included in each maximum value detection period is not particularly limited and can be set as appropriate.

The stain determination unit 23 determines whether or not the image pickup unit 10 is contaminated with time based on changes with time of the plurality of maximum values obtained for the plurality of maximum value detection periods.

More specifically, as illustrated in FIG. 3, the stain determination unit 23 stores the maximum value based on the time-series data of the maximum value (secondary maximum value) of the matching degree selected by the maximum value detection unit 22. When the value is equal to or less than a predetermined threshold stored in 30, it is determined that the imaging unit 10, particularly the optical member, is contaminated with time. In this case, the control unit 20 performs a notification process for notifying the administrator that the imaging unit 10, particularly the optical member, is contaminated with time.

Hereinafter, the operation flow of the face authentication apparatus according to the present embodiment will be described with reference to FIG.
First, the control unit 20 acquires an image photographed by the imaging unit 10, and stores it in the storage unit 30 as a reference image (step S1).

Next, after a lapse of a predetermined period from the acquisition of the reference image, the control unit 20 acquires an image captured by the imaging unit 10 and stores it as the input image 1 in the storage unit 30 (step S2, input image acquisition step). . Next, the coincidence calculation unit 21 calculates the coincidence of the input image 1 with respect to the reference image for each block and stores it in the storage unit 30 (step S3, coincidence calculation step). These processes are repeated every predetermined period, for example, every hour until the maximum value detection period elapses (step S4; No).

When the maximum value detection period elapses (step S4; Yes), the maximum value detection unit 22 matches the L input images 1, 2,..., L captured in the maximum value detection period. The maximum value is detected from the degree and stored in the storage unit 30 as time series data (step S5, maximum value detection step).

Next, the stain determination unit 23 determines whether or not the maximum value of the degree of coincidence detected by the maximum value detection unit 22 is equal to or less than a predetermined threshold (step S6, stain determination step), and the maximum value is a predetermined value. If it is not less than the threshold value, the steps from the input image acquisition step S2 to the stain determination step S6 are repeated (step S6; No). When the maximum value is equal to or less than the predetermined threshold (Step S6; Yes), the control unit 20 performs a notification process for notifying the administrator that the imaging unit 10, particularly the optical member, is contaminated with time. Perform (step S7). At this time, together with the notification process, a series of processes related to face authentication may be stopped, and the face authentication apparatus according to the present embodiment may be stopped.

As described above, in the embodiment, the partial reference image of the reference image captured by the imaging unit 10 and the partial input image corresponding to each partial reference image of the plurality of input images captured by the imaging unit 10 are obtained. For comparison, the degree-of-matching calculation unit 21 (the degree-of-matching calculation step S3) that calculates the degree of matching of each partial input image with respect to the partial reference image and each of a plurality of maximum value detection periods whose start times are before and after each other A maximum value detection unit 22 (maximum value detection step S5) for obtaining a maximum value among a plurality of matching degrees for a plurality of partial input images of a plurality of input images photographed within, and a plurality of maximum value detection periods. Since it includes a dirt determination unit 23 (dirt determination step S6) for determining whether or not the imaging unit 10 is contaminated over time based on the plurality of maximum values obtained over time, the imaging is performed. Part 1 In particular, the variation in the degree of coincidence due to the contamination of the optical member over time, the variation in the degree of coincidence due to changes in the background of the passerby, etc. It can be detected separately from the fluctuation of the degree. Therefore, in the above-described embodiment, the time-dependent dirt of the imaging unit 10 can be clearly expressed by a numerical value, and the administrator can be encouraged to clean the time-stained dirt of the imaging unit 10 at an appropriate timing. As a result, it is possible to prevent the face authentication apparatus according to the above embodiment from pausing, and to prevent the face authentication accuracy from being deteriorated due to the aging of the optical member. Can do.

Further, in the above-described embodiment, as described above, the presence / absence of contamination of the imaging unit 10 over time is determined based on the change over time of the maximum value of the plurality of coincidences for the plurality of partial input images. Therefore, unlike the devices described in Patent Documents 1 and 2, the reference image and / or each input image may be an image of a person to be authenticated, a place with a lot of traffic, or a place with a large background change. Even if it is an image of a place where the lighting environment changes depending on the time and changes the direction of the shadow, such as outdoors or near a window, it is possible to detect the temporal contamination of the imaging unit 10 separately from other factors. Can do.

In the above embodiment, as described above, the degree of coincidence of each partial input image with respect to the partial reference image is calculated. Therefore, unlike the apparatus described in Patent Document 3, the pattern (shape) of the partial reference image is calculated. The similarity of the patterns (shapes) of the partial input images can be detected, and the temporal contamination of the imaging unit 10 can be detected separately from the change in hardware characteristics.

In the above-described embodiment, the degree-of-match calculation unit 21 normalizes each of the partial reference image and the plurality of partial input images with brightness information included in the partial image, and normalizes the partial reference image and the plurality of input images. Since the degree of coincidence is calculated based on the image, information relating to the pattern can be extracted from each image data, and the degree of coincidence can be calculated based on the information relating to the pattern. The degree of coincidence can be calculated in a state where the influence of a change in brightness such as a change is reduced.

In the above embodiment, the coincidence calculation unit 21 calculates the coincidence based on the contour information included in each of the partial reference image and the plurality of partial input images. The degree of coincidence can be calculated in a state where the influence of a change in brightness such as a change in illuminance is reduced.

In the embodiment described above, the coincidence calculation unit 21 divides the reference image and the input image into a plurality of blocks corresponding to each other, and calculates the coincidence for each block, thereby calculating the coincidence for each input image as a whole. Therefore, it is possible to further reduce the influence of a temporary image change caused by a moving object such as a person to be authenticated or a passerby. That is, it is possible to detect the time-dependent dirt of the imaging unit 10 with higher accuracy.

Moreover, in the said embodiment, the maximum value detection part 22 selects the largest coincidence degree (primary maximum value) for every block from the several coincidence degree about the several input image image | photographed within the maximum value calculation period, Since the maximum value (secondary maximum value) is obtained from the maximum degree of coincidence (primary maximum value) for all of the blocks, when the imaging unit 10, particularly the optical member, becomes dirty as a whole, the contamination is effective. Can be detected.

In the above embodiment, the maximum value detection unit 22 compares the matching degrees of the L input images included in each maximum value calculation period for each block, and determines the maximum matching degree (primary maximum value) in each block. The maximum value (secondary maximum value) is selected from the matching degrees (M × N primary maximum values) of all blocks, and the maximum value detection unit 22 is included in each maximum value calculation period. In each of the L input images, the maximum (primary maximum value) is selected from the M × N matching degrees, and the matching value (L primary maximum value) of all the input images is selected to the maximum value (2 (Next maximum value) may be selected. Further, the maximum value detection unit 22 may select a maximum value from L × M × N coincidence degrees regarding L input images included in each maximum value calculation period.

As described above, the maximum value detection unit 22 obtains the maximum value from all the coincidence degrees calculated by the coincidence degree calculation unit 21 by dividing each of the plurality of input images taken during the maximum value calculation period into a plurality of blocks. Therefore, when the imaging unit 10, particularly the optical member, becomes dirty as a whole, the stain can be detected effectively.

In the above embodiment, the degree of coincidence is calculated for all the blocks, and the maximum value is detected from them. However, as shown in FIG. 5, the degree of coincidence is calculated only for some blocks, for example, some blocks that are easily dirty. Then, the maximum value may be detected from them.

In the above embodiment, the maximum value detection unit 22 obtains a maximum value for each of a plurality of maximum value detection periods, and the stain determination unit 23 captures an image for each of the plurality of maximum value detection periods. Since the presence / absence of the time-dependent contamination of the unit 10 is determined, the time-dependent contamination of the imaging unit 10 can be detected at an early stage.

Further, in the above embodiment, the dirt determination unit 23 determines whether or not the time-dependent dirt is generated by comparing the maximum value with a predetermined threshold value. Can be detected based on clear criteria.

(Embodiment 2)
In the present embodiment, features unique to the present embodiment will be mainly described, and the description overlapping with the first embodiment will be omitted. Moreover, in this embodiment and Embodiment 1, the same code | symbol is attached | subjected to the member which has the same or the same function, and description of the member is abbreviate | omitted in this embodiment. As described below, this embodiment is substantially the same as Embodiment 1 except that it detects stains over time of the imaging unit based on changes over time in the maximum value of the degree of coincidence for each block. It is.

As shown in FIG. 6, in the present embodiment, the maximum value detection unit 22, as in the case of the first embodiment, every time the maximum value detection period elapses, the L number of images captured by the imaging unit 10 throughout that period. The matching degree of the input image is compared for each block, and the maximum matching degree (maximum value) is selected for each block. On the other hand, the maximum value detection unit 22 then stores the degree of coincidence (M × N maximum values) of each block in the storage unit 30 as time series data, and unlike the case of the first embodiment, the degree of coincidence of all blocks. No further maximum value is obtained from (M × N maximum values).

Further, unlike the case of the first embodiment, the stain determination unit 23 is based on the time series data of the maximum value of the degree of coincidence for each block selected by the maximum value detection unit 22, as shown in FIG. When the maximum value related to the block is equal to or less than a predetermined threshold stored in the storage unit 30, it is determined that the imaging unit 10, particularly the optical member, is contaminated with time. In addition, the threshold value which concerns on each block may be the same, and may mutually differ.

As described above, in the present embodiment, the maximum value detection unit 22 includes a plurality of coincidence degrees calculated by the coincidence degree calculation unit 21 by dividing each of the plurality of input images taken during the maximum value detection period into a plurality of blocks. The maximum value is obtained for each corresponding block from the above, and the stain determination unit 23 causes the image pickup unit 10 to collect the time-dependent stain based on the temporal change of the plurality of maximum values for each block obtained for the plurality of maximum value detection periods. Therefore, when the imaging unit 10, particularly the optical member, is partially soiled, for example, the left side portion of the optical member is clean due to the wind direction, but the right side portion is In the case where it is dirty, the stain can be detected effectively.

(Embodiment 3)
In the present embodiment, features unique to the present embodiment will be mainly described, and the description overlapping with the first embodiment will be omitted. Moreover, in this embodiment and Embodiment 1, the same code | symbol is attached | subjected to the member which has the same or the same function, and description of the member is abbreviate | omitted in this embodiment. The present embodiment is substantially the same as the first embodiment except for the points described below.

As shown in FIG. 8, in this embodiment, the coincidence calculation unit 21 includes a partial reference image of each of the plurality of reference images 1,..., K taken by the imaging unit 10, and each partial input image. Are compared to calculate the degree of coincidence. That is, the coincidence calculation unit 21 calculates the coincidence of each input image with respect to each reference image 1,. As a result, a plurality of reference images taken in different situations can be used, and the possibility of moving objects such as passers-by appearing in the reference image can be reduced. It can be detected well.

(Embodiment 4)
In the present embodiment, features unique to the present embodiment will be mainly described, and the description overlapping with the first embodiment will be omitted. Moreover, in this embodiment and Embodiment 1, the same code | symbol is attached | subjected to the member which has the same or the same function, and description of the member is abbreviate | omitted in this embodiment. The present embodiment is substantially the same as the first embodiment except for the points described below.

As shown in FIG. 9, in the present embodiment, the control unit 20 further includes a comparison extraction unit 24 that compares the reference image with each input image and extracts a region with a small change. As shown in FIG. 10, the comparison extraction unit 24 compares the reference image with the input image and extracts a region 43 with a small change (difference). A region 43 corresponds to a background image. In the present embodiment, the degree-of-match calculation unit 21 compares the partial reference image with the partial input image for the entire region 43 or a part of the region 43 to match the partial input image with the partial reference image. Calculate the degree. According to the present embodiment, it is possible to remove in advance a region where the degree of coincidence is low, so that the amount of calculation for calculating the degree of coincidence can be reduced.

More specifically, the comparison and extraction unit 24 generates a difference image between the reference image and each input image, compares the pixel value of the difference image with a predetermined threshold value, and determines pixels that have a small difference and fall below the threshold value. It detects as area 43. Alternatively, the comparison / extraction unit 24 calculates a difference in luminance value or a distance in the color space for each pixel corresponding to the reference image and each input image, compares the calculated value with a predetermined threshold value, and the difference is small and below the threshold value. A pixel is detected as a region 43. The degree of coincidence calculation unit 21 may calculate the degree of coincidence for each block by dividing each region 43 into blocks in the same manner as in the first embodiment, or the whole region 43 without dividing each region 43 into blocks. One degree of coincidence may be calculated with

As described above, the present embodiment further includes the comparison extraction unit 24 (comparison extraction step) that compares the reference image and each of the plurality of input images and extracts the region 43 with a small change, and the matching degree calculation unit 21. Since the degree of coincidence is calculated in the region 43 having a small difference, the amount of calculation for calculating the degree of coincidence can be reduced.

(Deformation)
In the above embodiment, an example is described in which a plurality of maximum value detection periods are set in advance and are constant (periods of the same length), but each maximum value detection period is appropriately changed during the operation of the face authentication apparatus. The periods may be different from each other.

In the above embodiment, an example in which a plurality of maximum value detection periods are continuous without interruption has been described. However, the plurality of maximum value detection periods may overlap each other (for example, each maximum value detection period is 3 days). These periods may overlap one day at a time), or a maximum value detection period may be provided intermittently (for example, a maximum value detection period may be provided every other day).

In the above-described embodiment, an example has been described in which one maximum value, that is, the latest maximum value, is threshold-processed to determine whether or not the imaging unit 10 is contaminated over time. The criteria for contamination are not particularly limited and can be set as appropriate. For example, when a plurality of continuous maximum values are equal to or less than the threshold value, it may be determined that contamination with time has occurred. In addition, a simple moving average is calculated for each predetermined period (for example, every three days) from the time series data of the maximum value, and if one or a plurality of calculated simple moving average values are equal to or less than a threshold value, contamination over time It may be determined that has occurred. Further, when one or a plurality of continuous maximum values are less than or equal to the threshold value, the notification process is performed once. After that, when one or a plurality of continuous simple moving average values are equal to or less than the threshold value, the notification process is performed again. Also good. At this time, the first threshold and the second threshold may be the same or different from each other. In addition, an approximate curve is calculated not from the maximum value or its average value but from a plurality of continuous maximum values, and based on the magnitude of the slope of the approximate curve and its positive / negative, the contamination of the imaging unit 10 over time is calculated. The presence or absence of occurrence may be determined. This approximate curve may be an approximate line. Further, a change amount of the maximum value such as a differential value of the approximate curve (approximate straight line) is obtained, and notification is given when a period in which the magnitude of the change amount is greater than a predetermined threshold continues for a predetermined period or longer. Also good. As a result, even if the environment changes and the stain of the imaging unit 10 progresses with time, and measures such as cleaning are required earlier than the scheduled time, it is possible to cope with it.

In the above embodiment, an example in which the imaging apparatus, the dirt detection system, and the dirt detection method according to the present invention are applied to a face authentication apparatus has been described. However, the present invention is an image collation that collates two or more images. The present invention can be applied to all devices, image collation systems, and image collation devices in general. For example, the present invention may be applied to an image collation apparatus that shows a point where a person is present and a movement route of the person based on a plurality of person images photographed by a plurality of imaging units. Furthermore, the present invention can also be applied to uses that do not necessarily require image collation, for example, surveillance cameras in general such as security cameras.

As mentioned above, although embodiment of this invention was described referring drawings, this invention is not limited to the said embodiment. In addition, the configurations of the respective embodiments may be appropriately combined or changed within a range not departing from the gist of the present invention.

As described above, the present invention is a useful technique for detecting the time-dependent contamination of the imaging unit.

10: Imaging unit 20: Control unit 21: Matching degree calculation unit 22: Maximum value detection unit 23: Dirt determination unit 24: Comparison extraction unit 30: Storage unit 41: Partial reference image 42: Partial input image 43: Change (difference) Small area

Claims (12)

An imaging device capable of detecting contamination over time of an imaging unit,
An imaging unit;
The partial reference image of the reference image captured by the imaging unit is compared with the partial input image corresponding to the partial reference image of each of the plurality of input images captured by the imaging unit, and the partial reference image A degree of coincidence calculation unit for calculating the degree of coincidence of each partial input image;
For each of a plurality of periods before and after the start time of each other, a maximum value detection unit for obtaining a maximum value among a plurality of coincidences for a plurality of partial input images of a plurality of input images photographed within the period;
A stain determination unit that determines whether or not stains over time have occurred in the imaging unit, based on changes over time of a plurality of maximum values obtained for the plurality of periods;
An imaging apparatus comprising: The degree of coincidence calculation unit normalizes each of the partial reference image and the plurality of partial input images with light and dark information included in the partial image, and normalizes the partial reference image and the plurality of normalized partial inputs. The imaging device according to claim 1, wherein the degree of coincidence is calculated based on an image. The imaging apparatus according to claim 1, wherein the coincidence degree calculation unit calculates the coincidence degree based on contour information included in each of the partial reference image and the plurality of partial input images. The degree of coincidence calculation unit divides the reference image and the input image into a plurality of partial areas corresponding to each other, and calculates the degree of coincidence for each partial area. The imaging device described in 1. The maximum value detecting unit obtains the maximum value from all the coincidence degrees calculated by the coincidence degree calculating unit by dividing each of the plurality of input images photographed within the period into a plurality of partial areas. The imaging device according to claim 4. The maximum value detection unit is configured to output the plurality of input images captured during the period by dividing the plurality of input images into a plurality of partial regions for each partial region corresponding to the corresponding partial regions. Find the maximum value
The dirt determination unit determines, for each partial area, whether or not the time-dependent dirt has occurred, based on a change with time of a plurality of maximum values for each of the partial areas obtained for the plurality of periods. The imaging apparatus according to claim 4, wherein: A comparison extraction unit that compares the reference image with each of the plurality of input images and extracts a region with a small change;
The imaging device according to claim 1, wherein the coincidence calculation unit calculates the coincidence in an area where the difference is small. 2. The coincidence degree calculation unit calculates the degree of coincidence by comparing each partial reference image of each of a plurality of reference images taken by the imaging unit and each partial input image. The imaging device according to any one of? The maximum value detection unit obtains the maximum value for each progress of the plurality of periods,
The imaging device according to claim 1, wherein the stain determination unit determines whether or not the stain with time is generated for each of the plurality of periods. The imaging device according to any one of claims 1 to 9, wherein the stain determination unit determines whether the stain with time has occurred or not by comparing the maximum value with a predetermined threshold value. . A dirt detection system capable of detecting dirt over time of an imaging unit,
The partial reference image of the reference image captured by the imaging unit is compared with the partial input image corresponding to the partial reference image of each of the plurality of input images captured by the imaging unit. A degree-of-matching calculation unit that calculates the degree of matching of the partial input images;
For each of a plurality of periods before and after the start time of each other, a maximum value detection unit for obtaining a maximum value among a plurality of coincidences for a plurality of partial input images of a plurality of input images photographed within the period;
A stain determination unit that determines whether or not stains over time have occurred in the imaging unit, based on changes over time of a plurality of maximum values obtained for the plurality of periods;
A dirt detection system comprising: A contamination detection method for detecting contamination over time of an imaging unit,
The partial reference image of the reference image captured by the imaging unit is compared with the partial input image corresponding to the partial reference image of each of the plurality of input images captured by the imaging unit. A degree-of-match calculation step for calculating the degree of match of the partial input images;
A maximum value detecting step for obtaining a maximum value of a plurality of coincidences for a plurality of partial input images of a plurality of input images photographed within the period for each of a plurality of periods whose start times are around each other;
A stain determination step for determining whether or not stains over time have occurred in the imaging unit based on changes over time of a plurality of maximum values obtained for the plurality of periods;
A dirt detection method comprising:

Images