JP2017062633A - Image collation device and image collation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To conduct correct and efficient image collation even when a mode of an object included in input image data and registration image data is different.SOLUTION: When a front face image of a person is included in registration image data A1, and a front face image of a person is included in input image data B1, a part of a front face upper part shown by a slant line in FIG. is made more weighted as a characteristic part. When the front face image of the person is included in the registration image data A1, and a lateral face image of the person is included in input image data B2, a characteristic part of the person shown by a white area in FIG. is made less weighted. Accordingly, on the basis of a direction of the person in the registration image data A and the input image data B, the weight of the characteristic part is varied to thereby conduct correct collation processing with the characteristic part taken into consideration.SELECTED DRAWING: Figure 1

Description

  The present invention provides an image for accurately and efficiently performing image matching when the object such as a person or vehicle included in the input image data photographed by a monitoring camera or the like and the object included in the registered image data are different from each other. The present invention relates to a collation device and an image collation method.

  Conventionally, a technique for detecting a specific object from input image data by comparing an object included in input image data with an object included in registered image data registered in advance using color features is known. It has been.

  For example, in Patent Document 1, the entire region of an object is divided into regions for each height, the colors of the divided regions are extracted and averaged, and based on the averaged region colors for each height. There is an object identification method that creates a color feature distribution for each height of the entire object and identifies the object to be compared with the object based on the comparison of the color feature distribution of each object with the color feature distribution of the object to be compared. It is disclosed.

  Further, in Patent Document 2, information related to the clothing characteristics of a person is extracted based on the registered image data photographed by the first photographing means and stored in a database. In other environments, one or two or more second A person search system is disclosed that extracts information on clothes characteristics of a person based on input image data photographed by the photographing means and searches a predetermined person included in the input image data by collating with the database. .

Japanese Patent Laid-Open No. 2005-250692 JP 2013-186546 A

  However, in the related art represented by Patent Documents 1 and 2, when the orientation and orientation of the person included in the input image data are different from the orientation and orientation of the person included in the registered image data, the person is accurately identified. May not be identified. This is because the color feature distribution and clothing features differ if the orientation and posture of the person differ.

  In addition, if the person included in the input image data has personal belongings such as a bag or a cane but the person included in the registered image data does not have personal belongings, the person is the same person. However, there may be a difference in color feature distribution and clothing features, and it may be erroneously determined that the person is another person.

  For these reasons, it is an important issue how to perform image matching accurately and efficiently when the input image data and registered image data have different orientations, postures, belongings, etc. Yes. Such a problem is not only a problem that occurs only when a person is targeted, but also a problem that occurs when another object such as a vehicle is targeted.

  The present invention has been made in order to solve the above-described problems of the prior art, and is capable of accurately and efficiently performing image verification even when input image data and registered image data have different object forms. An object of the present invention is to provide an image collation apparatus and an image collation method capable of performing the above.

  In order to solve the above-described problems, the present invention provides an image collation apparatus that collates input image data in which a predetermined object is reflected and registered image data registered in advance, and the input image data and / or Alternatively, based on the correspondence relationship between the input image data and the registered image data, the weight target region specifying unit that divides the target object region of the registered image data and specifies the weight target region that forms a part of the divided region. A weight determination unit that determines a weight to be given to the weight target region, and a verification that applies the weight determined by the weight determination unit to the weight target region and performs a verification process on the input image data and the registered image data And a processing unit.

  Also, in the present invention according to the above invention, the weight determination unit may be configured such that the aspect of the object included in the input image data and the aspect of the object included in the registered image data match or are similar to each other. , Increase the weight to be given to the weight target area, and if the aspect of the target object included in the input image data and the aspect of the target object included in the registered image data are not similar, give to the weight target area It is characterized by lowering the weight to be performed.

  Further, the present invention is the above invention, wherein the weight determination unit, when the direction of the object included in the input image data and the direction of the object included in the registered image data match, The weight given to the weight target area is increased when the weight given to the weight target area is increased, and the direction of the target included in the input image data does not match the direction of the target included in the registered image data It is characterized by lowering.

  Also, in the present invention according to the above invention, the weight determination unit includes a feature in the object included in the input image data, and the feature in the object included in the registered image data. If there is a feature in one of the input image data or the registered image data, the weight to be given to the weight target area is lowered. It is characterized by.

  Further, according to the present invention, in the above invention, the collation processing unit applies a weighted histogram of the input image data and the registered image data while applying the weight determined by the weight determination unit to the weight target region. The input image data and the registered image data are collated based on the calculated weighted histograms.

  Further, according to the present invention, in the above invention, the collation processing unit divides the input image data and the registered image data into small regions having similar colors or shades, and defines the small regions corresponding to the weight target regions. On the other hand, the small area of the input image data and the small area of the registered image data are collated while applying the weight determined by the weight determining unit.

  Further, the present invention is an image collating method in an image collating apparatus for collating input image data in which a predetermined object is reflected and registered image data registered in advance, wherein the input image data and / or the input image data Based on the correspondence relationship between the input image data and the registered image data, the weight target region specifying step for dividing the region of the object of the registered image data and specifying the weight target region that forms a part of the divided region, A weight determining step for determining a weight to be given to the weight target region; and a collation processing step for performing the collation processing on the input image data and the registered image data while applying the weight determined by the weight determining unit to the weight target region. It is characterized by including.

  According to the present invention, there is provided an image collation apparatus that collates input image data in which a predetermined object is reflected and registered image data registered in advance, wherein the input image data and / or the registered image data A weight target area specifying unit for specifying a part of the weight target area; a weight determining unit for determining a weight to be given to the weight target area based on a correspondence relationship between the input image data and the registered image data; The input image and the registered image are included because the input image data and the registered image data are collated while applying the weight determined by the weight determining unit to the weight target area. Even if the object has a different aspect, image collation can be performed accurately and efficiently.

FIG. 1 is an explanatory diagram for explaining the features of the image matching apparatus according to the first embodiment. FIG. 2 is a functional block diagram illustrating the configuration of the image collating apparatus according to the first embodiment. FIG. 3 is a flowchart showing a processing procedure of the image collating apparatus shown in FIG. FIG. 4 is a flowchart showing a processing procedure of the preprocessing shown in FIG. FIG. 5 is an explanatory diagram for explaining the preprocessing shown in FIG. FIG. 6 is an explanatory diagram for explaining the flattening of the histogram shown in FIG. FIG. 7 is an explanatory diagram for explaining the concept of histogram flattening. FIG. 8 is a flowchart showing a processing procedure of the weight setting process shown in FIG. FIG. 9 is an explanatory diagram for estimating a part by the weight target area specifying unit. FIG. 10 is an explanatory diagram for specifying the weight target area. FIG. 11 is an explanatory diagram for determining the weight by the weight determination unit. FIG. 12 is a flowchart showing a processing procedure for calculating the matching score S _HSV shown in FIG. FIG. 13 is an explanatory diagram for calculating the matching score S _HSV . FIG. 14 is a flowchart showing a processing procedure for calculating the matching score S _SEG shown in FIG. FIG. 15 is an explanatory diagram for calculating the matching score S _SEG . FIG. 16 is an explanatory diagram for explaining the features of the image matching apparatus according to the second embodiment. FIG. 17 is a functional block diagram illustrating the configuration of the image collating apparatus according to the second embodiment. FIG. 18 is a flowchart showing a processing procedure of the image collating apparatus shown in FIG.

  Embodiments of an image matching apparatus and an image matching method according to the present invention will be described below with reference to the accompanying drawings. In the first embodiment described below, a case where weights are determined using the orientations of persons included in registered image data and input image data will be described. In the second embodiment, registered image data and input image data A case will be described in which the weight is determined using the personal belongings included in each.

  First, features of the image collating apparatus according to the first embodiment will be described. FIG. 1 is an explanatory diagram for explaining the features of the image matching apparatus according to the first embodiment. Here, a case is shown in which the registered image data A and the input image data B are collated using the degree of matching of appearance colors such as a person's clothes. This is because there may be cases where image data having sufficient resolution cannot be obtained from the image data acquired from the surveillance camera, and collation processing based on detailed shape features such as face collation may not be performed.

  As shown in FIG. 1A, when the registered image data A1 includes a front image of a person and the input image data B1 includes a front image of a person, the person indicated by diagonal lines in the figure. The weight is increased with the upper part of the front as a characteristic part. This is because the upper part of the front of the person indicated by hatching in the figure often has a characteristic hue. However, a portion other than the upper front portion of the person can be used as the characteristic portion.

  On the other hand, as shown in FIG. 1B, when the registered image data A1 includes a front image of a person and the input image data B2 includes a side image of a person, The weight of the characteristic part of the person indicated by is reduced. This is because the upper front part of the person does not appear in the input image data B2.

  As described above, the registered image data A and the input image data B can be accurately collated in consideration of the feature portion by changing the weight of the feature portion based on the orientation of the person.

  Next, the configuration of the image matching device according to the first embodiment will be described. FIG. 2 is a functional block diagram illustrating the configuration of the image collating apparatus according to the first embodiment. As shown in FIG. 2, the image matching device 20 is connected to the camera 10, the operation unit 30, and the display unit 40.

  The camera 10 is an imaging device such as a surveillance camera that images the surroundings. The image collation device 20 cuts out a region including a human image from the image data captured by the camera 10 and uses it as input image data B. The number of cameras 10 is not necessarily one, and a plurality of imaging devices can be used as each camera 10.

  The operation unit 30 is an input interface that receives an operation input from an operator, such as a keyboard, a mouse, and a dedicated operation panel. The display unit 40 is an output interface that performs display output of a liquid crystal display or the like.

  The image matching device 20 includes a storage unit 21 and a control unit 22 therein. The storage unit 21 is a storage device such as a hard disk device or a nonvolatile memory, and stores registration data 21a. The storage unit 21 is used to store a candidate image list 21b generated in the verification process.

  The registered data 21a has a plurality of registered image data A, and each registered image data A is associated with orientation data. The plurality of registered image data A are generated for different persons, for example, and one registered image data A includes one person image. The orientation data indicates the orientation of the person in the corresponding registered image data A. FIG. 2 shows a state in which the orientation data A1a is associated with the registered image data A1.

  The candidate image list 21b is a list of registered image data A whose similarity with the input image data B is equal to or greater than a threshold value in the collation described later. In other words, the candidate image list 21b indicates candidates for a person who may match the person shown in the input image data B.

  The control unit 22 is a control unit that controls the entire image collating apparatus 20, and includes a pre-processing unit 22a, a weight target region specifying unit 22b, a weight determining unit 22c, a total score calculating unit 22d, a determination processing unit 22e, and a display control unit 22f. Have. Actually, programs corresponding to these functional units are stored in a ROM or a non-volatile memory (not shown), and these programs are loaded into a CPU (Central Processing Unit) and executed, whereby the preprocessing unit 22a, Processes corresponding to the weight target area specifying unit 22b, the weight determining unit 22c, the total score calculating unit 22d, the determination processing unit 22e, and the display control unit 22f are executed.

  The preprocessing unit 22a is a processing unit that generates input image data B from imaged data captured by the camera 10 as preprocessing for collation. Specifically, the pre-processing unit 22a detects a person image from the imaging data, estimates the direction of the person, and cuts out an area including the person image as a person area. Then, the size of the cut-out human region is normalized, and the input image data B is generated by flattening the pixel histogram.

  The weight target area specifying unit 22b is a processing unit that estimates a human part of a person image included in the registered image data A and the input image data B, and specifies a weight target area from the estimated part. As the weight target region, a feature portion including many features of the person among the portions of the person is used. For example, the upper front part of the person is a feature part including many features of the person, and this feature part is set as a weight target region.

  The weight determination unit 22c is a processing unit that determines the weight to be given to the weight target region based on the correspondence relationship between the registered image data A and the input image data B. Here, the weight determination unit 22c uses the orientation of the person for the correspondence between the registered image data A and the input image data B.

  Specifically, the weight determination unit 22c sets the weight target area when the orientation of the person estimated for the input image data B matches the orientation indicated in the orientation data corresponding to the registered image data A. Increase the weight to be given. On the other hand, if the orientation of the person estimated for the input image data B does not match the orientation indicated in the orientation data corresponding to the registered image data A, the weight assigned to the weight target area is lowered.

  The total score calculation unit 22d applies the weight determined by the weight determination unit 22c to the weight target region, collates the input image data B and the registered image data A based on the applied weight, and calculates a total score indicating the degree of matching. It is a processing unit.

Specifically, the total score calculation unit 22d has a matching score S _HSV that is an evaluation value obtained by evaluating the degree of matching for each part of the person and a matching score that is an evaluation value obtained by evaluating the degree of matching for each small region having a similar color. S _SEG is calculated, and the total of the matching score S _HSV and the matching score S _SEG is calculated as a total score. Specific calculation processing of the matching score S _HSV and the matching score S _SEG will be described later.

  The determination processing unit 22e compares the total score with the threshold value to determine whether or not there is a high possibility that the person shown in the input image data B and the person shown in the registered image data A are the same person. It is a processing part to determine. As a result of the determination, the determination processing unit 22e registers the registered image data A whose total score is equal to or greater than the threshold value in the candidate image list 21b.

  The display control unit 22f is a processing unit that controls the display of the registered image data A registered in the candidate image list 21b on the display unit 40. The operator can confirm a candidate for a person that matches the person shown in the input image data B by this display control.

  Next, the processing procedure of the image collation apparatus 20 will be described. FIG. 3 is a flowchart showing a processing procedure of the image collating apparatus 20. First, when imaging data is received from the camera 10 (step S101), the preprocessing unit 22a of the image collating device 20 performs preprocessing on the imaging data to generate input image data B (step S102).

  Further, the determination processing unit 22e selects one of the plurality of registered image data A (step S103). The weight target area specifying unit 22b and the weight determining unit 22c perform weight setting processing using the input image data B generated in step S102 and the registered image data A selected in step S103 (step S104).

The total score calculation unit 22d collates the input image data B and the registered image data A for each part of the person while applying the weight set in step S104 to the weight target region, and calculates a collation score S _HSV (step S105). ).

Further, the total score calculation unit 22d collates the input image data B and the registered image data A for each small region having similar colors while applying the weight set in step S104 to the weight target region, and _obtains the collation score S _SEG . Calculate (step S106).

The total score calculation unit 22d calculates the total score S by summing the matching score S _HSV and the matching score S _SEG (step S107). The determination processing unit 22e compares the total score S with the threshold value T (step S107).

  As a result, if the total score S is equal to or greater than the threshold value T (step S108; Yes), the determination processing unit 22e registers the registered image data A selected in step S103 in the candidate image list 21b (step S109). .

  After step S109 or when the total score S is less than the threshold value T (step S108; No), the determination processing unit 22e determines whether or not unprocessed registered image data A that has not been selected in step S103 remains. Is determined (step S110). If unprocessed registered image data A remains (step S110; Yes), the determination processing unit 22e proceeds to step S103 and selects one of the unprocessed registered image data A.

  If unprocessed registered image data A does not remain (step S110; No), the display control unit 22f displays the registered image data A registered in the candidate image list 21b on the display unit 40 (step S111). Exit.

  Next, the preprocessing shown in step S102 of FIG. 3 will be described in detail. FIG. 4 is a flowchart showing a processing procedure of the preprocessing shown in FIG. First, the preprocessing unit 22a detects an image of a person from the imaging data (step S201), and estimates the direction of the person (step S202). For detection of a person's image and estimation of a person's orientation, for example, a plurality of templates each modeling the shape of a person with a different orientation may be prepared, and a target may be found by pattern matching. As a pattern matching method, an arbitrary method such as machine learning such as SVM or a subspace method can be used.

  The pre-processing unit 22a cuts out a region including the detected human image as a human region (step S203), normalizes the size of the cut-out human region (step S204), and flattens the pixel histogram (step S205). . The image data obtained by these processes is the input image data B. After step S205, the preprocessing unit 22a ends the preprocessing and returns to the process of FIG.

  FIG. 5 is an explanatory diagram for explaining the preprocessing shown in FIG. The image data C1 illustrated in FIG. 5 corresponds to the image data captured by the camera 10. Image data C2 is obtained by cutting out a person area from the image data C1 by pattern matching or the like and normalizing the size. The pre-processing unit 22a generates the image data C3 by removing the background from the image data C2, and further generates the image data C4 by flattening the pixel histogram in the image data C3. This image data C4 corresponds to the input image data B.

  The histogram flattening by the preprocessing unit 22a will be further described. FIG. 6 is an explanatory diagram for explaining the flattening of the histogram shown in FIG. The preprocessing unit 22a first performs HSV conversion on the image data C3 when the histogram is flattened. By this HSV conversion, image data C3h indicating the hue component of the image data C3, image data C3s indicating the saturation component of the image data C3, and image data C3v indicating the brightness component of the image data C3 are obtained.

ここで、Ｖは変換前の画素値であり、Ｖ’は変換後の画素値である。また、ＨｉｓｔはＶ成分のヒストグラムであり、ＳＩＺＥは人物領域の画素数である。 The preprocessing unit 22a performs the following conversion on the pixels in the person area of the image data C3v to generate image data C3v ′.

Here, V is a pixel value before conversion, and V ′ is a pixel value after conversion. Hist is a histogram of the V component, and SIZE is the number of pixels in the person area.

  Thereafter, the preprocessing unit 22a generates image data C4 corresponding to the input image data B by converting the image data C3h, the image data C3s, and the image data C3v 'into RGB images.

  FIG. 7 is an explanatory diagram for explaining the concept of histogram flattening. FIG. 7A is a histogram of lightness components in the image data C3v before flattening, and FIG. 7B is a histogram of lightness components in the image data C3v ′ after flattening. Comparing before and after flattening, the lightness value x is biased to a narrow range before flattening, whereas the lightness value x is distributed evenly after flattening.

  By flattening the histogram in this way, it is possible to eliminate the unevenness of brightness in the input image data B and to align the brightness between the input image data B and the registered image data A.

  Next, the weight setting process shown in step S104 of FIG. 3 will be described in detail. FIG. 8 is a flowchart showing a processing procedure of the weight setting process shown in FIG. First, the weight target area specifying unit 22b estimates a human part for the human image included in the registered image data A and the input image data B (step S301). Furthermore, the weight target area specifying unit 22b specifies the weight target area from the estimated part (step S302).

  The weight determination unit 22c determines the weight to be given to the weight target region based on the orientation of the person estimated for the input image data B and the orientation indicated in the orientation data corresponding to the registered image data A (step S303). ). The weight setting process ends by the determination of the weight, and the process returns to the process of FIG.

  Next, the part estimation by the weight target area specifying unit 22b will be described. FIG. 9 is an explanatory diagram for estimating a part by the weight target area specifying unit 22b. First, the weight target area specifying unit 22b estimates a head, a body, a foot, and a central axis for a person image included in the registered image data A and the input image data B. In the present embodiment, “body” refers to a portion of the body excluding the head and feet. Moreover, it is good also as a part except a head and limbs in the body.

  FIG. 9A is an explanatory diagram of head, body and foot estimation. For the registered image data A and the input image data B, the weight target area specifying unit 22b calculates iTL indicating a foot-body boundary and iHLy indicating a body-head boundary, respectively. iTL and iHLy are obtained as values in the height direction of the registered image data A and the input image data B, that is, the Y axis.

When iTLy is calculated, the weight target region specifying unit 22b calculates the following feature F between σy1 and σy2 while scanning y within Δy.
F (y) = 1−C (y, σ) + S (y, σ)

  Here, Δy is a designated range in which a range where iTLy may be located in the registered image data A and the input image data B is designated in advance. Since the registered image data A and the input image data B are cut out as person regions, the boundary between the foot and the body is never located at an extremely high or low position. Therefore, the range where the foot-body boundary is not located is excluded and Δy is designated in advance.

  Further, σy1 and σy2 are partial regions set so that the registered image data A and the input image data B include an area that can detect a partial feature. The widths of σy1 and σy2 in the x-axis direction are equal to the widths of registered image data A and input image data B in the x-axis direction. Further, the height of σy1 in the y-axis direction is the same as the height of σy2 in the y-axis direction. The lower end of σy1 is the value of the variable y, and is set above the variable y. The upper end of σy2 is the value of the variable y, and is set below the variable y.

  C (y, σ) is the Euclidean distance between σy1 and σy2 in the HSV color space. S (y, σ) is the difference in area between the person portion in σy1 and the person portion in σy2. The weight target area specifying unit 22b obtains the y coordinate that minimizes F (y), and sets the y coordinate to iTLy.

When calculating iHL, the weight target area specifying unit 22b calculates the following feature F while scanning y within the specified range in the same manner as iTLy.
F (y) = S (y, σ)
Then, the y coordinate at which F (y) is maximized is obtained, and the y coordinate is set to iHL. Note that the designated range when iHL is calculated is a range in which the iHL may be located in advance.

  FIG. 9B is an explanatory diagram regarding the estimation of the central axis. The weight target area specifying unit 22b obtains a body center axis jLR1 and a foot center axis jLR2. When determining the center axis of the body, the target range is from iTL to iHLy, and when determining the center axis of the foot, the target range is from the lower end (y = 0) to iHLy. jLR1 and jLR2 are obtained as values in the horizontal direction in the registered image data A and the input image data B, that is, the x-axis.

When calculating the central axis, the weight target region specifying unit 22b calculates the following feature F between σx1 and σx2 while scanning x.
F (x) = C (x, σ) + S (x, σ)
Then, the x-coordinate at which F (x) is minimum is set as the central axis.

  Here, σx1 is a partial region on the left side of the variable x, and σx2 is a partial region on the right side of the variable x. FIG. 9B shows σx1 and σx2 when the center axis jLR2 of the foot is obtained. The height of σx1 and σx2 in the y-axis direction is equal to the height of the target range. Also, the width of σx1 in the x-axis direction and the width of σx2 in the x-axis direction are the same. This width is preferably set to include more than half of the person portion in the target range. Σx1 is the value of the variable x at the right end, and σx2 is the value of the variable x at the left end.

  The weight target area specifying unit 22b specifies the weight target area using the orientations of the registered image data A and the input image data B in addition to iTL, iHLy, jLR1, and jLR2. FIG. 10 is an explanatory diagram for specifying the weight target area.

  First, when the orientation of the person in the registered image data A and the input image data B is the front, the weight target area specifying unit 22b includes five areas: head, foot, left of the body, center of the body, and right of the body. Set. On the other hand, when the orientation of the person is the side or the back, three areas of the head, feet, and body are set.

  In FIG. 10A, the orientation of the person in the registered image data A2 is the front. For this reason, as shown in the area data PA2, five areas of the head, the foot, the left of the body, the center of the body, and the right of the body are set. On the other hand, in FIG. 10B, the orientation of the person in the input image data B3 is the back side. For this reason, as shown in the area data PB3, three areas of a head, a foot, and a body are set.

  Note that the position where the body part is divided into the left part of the body, the center of the body, and the right part of the body is a position separated by w from the center axis jLR1 of the body. Here, for w, an appropriate value is set in advance from the width of the image data and the normalized size of the person.

  The weight target area specifying unit 22b selects a weight target area from the set area. For example, the center of the body is a feature portion that includes many human features and is suitable as a weight target region. It is also possible to set all the set areas as weight target areas and set the weight for each area.

  Next, determination of the weight by the weight determination unit 22c will be described. FIG. 11 is an explanatory diagram for determining weights by the weight determination unit 22c. The weight determination unit 22c determines a weight based on a combination of person orientations in the registered image data A and the input image data B.

  FIG. 11 shows a correspondence relationship between combinations of person orientations and weights. 11 indicates that feature extraction is performed with emphasis by increasing the weight, and white region indicates that feature extraction is performed with the weight reduced.

  As illustrated in FIG. 11, the weight determination unit 22 c increases the weight of the center of the body when the combination of the person orientations is “front-front”. This is because there are many features in the center of the body due to the design of clothes and the wearing of ties and scarves. On the other hand, if the combination of person orientations is “front-side”, the weight of the center of the body is lowered. Further, when the combination of person orientations is “front-back”, the weight of the center of the body and the head are lowered. This is because the central feature of the body often does not appear on the side or back. Moreover, it is because the characteristics of a front, a side, and a back differ greatly. If the combination of the orientations of the persons is “side-back”, the weight of the head is lowered. Although illustration is omitted, no weight is given when the combination of the orientations of the persons is “back-back”.

  In FIG. 11, if the person's orientation is the front, the body part is divided into areas of the center and the left and right, and different weights are given. If the person's orientation is other than the front, the entire body part is the same as one area. Is weighted. This area division defines a range to which weight is given, and is not used as a unit of collation. That is, when collating the body part, the body part divided into three areas and the body part that is one area are collated.

  In this embodiment, three modes of standing on the front, side, and back are used and weights are determined for each combination. However, the modes to be used are not limited to three. For example, it is possible to further add a mode for a posture such as a diagonally forward left direction and a diagonally forward right direction and a sitting position. In that case, a template corresponding to each aspect is prepared and detected. Further, a weight is determined for each combination of modes.

  As described above, the weight determination unit 22c increases the weight if the feature portion is included in both the registered image data A and the input image data B, and decreases the weight if the feature portion is included in only one of them. If the feature part is included in both the registered image data A and the input image data B, the feature part contributes to the improvement of the collation accuracy, but the feature part included only in one side rather lowers the collation accuracy. This is because the collation accuracy can be expected to improve if attention is paid to areas other than the characteristic part.

Next, the calculation of the matching score S _HSV shown in step S105 of FIG. 3 will be described in detail. FIG. 12 is a flowchart showing a processing procedure for calculating the matching score S _HSV shown in FIG.

First, the total score calculation unit 22d calculates a weighted three-dimensional HSV histogram for each part of the registered image data A (step S401). Also, a weighted three-dimensional HSV histogram is calculated for each part of the input image data B (step S402). Then, the Bachata rear distance is calculated for each part, the average is set as the matching score S _HSV (step S403), and the process returns to FIG.

  Here, as the weight in calculating the three-dimensional HSV histogram, a weight obtained by multiplying the weight determined by the weight determining unit 22c and the weight based on the Gaussian distribution according to the distance from the central axis is used. When the orientation is the front, the body part is divided into three areas, but all areas are used when calculating the three-dimensional HSV histogram for the body part.

Further, the Bachata rear distance between the histograms h ₁ and h ₂ can be obtained by the following equation, where n is the number of bins in the histogram.

FIG. 13 is an explanatory diagram for calculating the matching score S _HSV . In FIG. 13, the orientation of the person in the registered image data A2 is the front, and the orientation of the person in the input image data B3 is the back. The total score calculation unit 22d uses the corresponding area data PA2 for weighting for the registered image data A2, and uses the corresponding area data PB3 for weighting for the input image data B3.

The total score calculation unit 22d applies weights based on the area data PA2 and the area data PB3 to the registered image data A2 and the input image data B3, respectively, and performs histogram matching for each part of the registered image data A2 and the input image data B3. The collation score S _HSV is calculated.

Next, the calculation of the matching score S _SEG shown in step S106 of FIG. 3 will be described in detail. FIG. 14 is a flowchart showing a processing procedure for calculating the matching score S _SEG shown in FIG.

First, the total score calculation unit 22d divides the registered image data A into color regions (step S501). The color area is a small area composed of continuous pixels with similar colors. Similarly, the total score calculation unit 22d divides the input image data B into color regions (step S502). Then, a collation score S _SEG between the color regions is calculated (step S503), and the process returns to the process of FIG.

ここで、Ｓ_seg1は、

であり、Ｓ_seg2は、

である。なお、Ｔｐは登録画像データＡから分割した色領域の集合であり、Ｉｎは入力画像データＢから分割した色領域の集合である。ｉ，ｊは、色領域の選択に用いる変数である。Ｓ_seg1の算出時にはｉ∈Ｉｎ、ｊ∈Ｔｐとし、Ｓ_seg2の算出時にはｉ∈Ｔｐ、ｊ∈Ｉｎとしている。すなわち、Ｓ_seg1及びＳ_seg2は、ｗij(γｄ^ij _y＋（１−γ）ｄ^ij _c)が最小となるｊを各ｉについて求めている。 The matching score S _SEG between the color areas is obtained by the following equation.

Where S _seg1 is

And S _seg2 is

It is. Tp is a set of color areas divided from the registered image data A, and In is a set of color areas divided from the input image data B. i and j are variables used to select a color region. When S _seg1 is calculated, iεIn and jεTp are set, and when S _seg2 is calculated, iεTp and jεIn are set. That is, S _seg1 and S _seg2 obtain j for each i where wij (γd ^ij _y + (1−γ) d ^ij _c ) is minimum.

  For the color area divided from the registered image data A, the center of gravity is gj, the average RGB value is cj, and the weight set for the area to which gj belongs is wj. Similarly, regarding the color area divided from the input image data B, the center of gravity is set to gi, the average RGB value is set to ci, and the weight set to the area to which gi belongs is set to wi.

d ^ij _y is the y-coordinate distance of the center of gravity,
d ^ij _y = | g _iy- g _jy |
Ask for.
d ^ij _c is the average RGB distance,
d ^ij _c = | c _i −c _j |
Ask for.

w _ij is the weight,
w _ij = 1 / min (w _i , w _j )
Is used. Note that γ is a predetermined parameter. Further, n _in is the sum of weights used _in S _seg1 , and n _tp is the sum of weights used in S _seg2 . Num _in is the number of color areas divided from the input image data B, and num _tp is the number of color areas divided from the registered image data A.

FIG. 15 is an explanatory diagram for calculating the matching score S _SEG . In FIG. 15, the orientation of the person in the registered image data A2 is the front, and the orientation of the person in the input image data B3 is the back. The total score calculation unit 22d divides the registered image data A2 into color areas to generate color area data QA2. Similarly, the input image data B3 is divided into color areas to generate color area data QB3.

  The total score calculation unit 22d uses the corresponding area data PA2 for weighting for the color area data QA2, and uses the corresponding area data PB3 for weighting for the color area data QB3.

The total score calculation unit 22d compares the color areas of the color area data QA2 and the color area data QB3 using the weights based on the area data PA2 and the area data PB3, and calculates a verification score S _SEG .

Next, the calculation of the total score shown in step S107 in FIG. 3 will be described. The total score calculation unit 22d uses the matching score S _HSV and the matching score S _SEG and calculates the total score S as S = 0.5 × S _HSV + 0.5 × S _SEG
Calculate as Here, a case has been exemplified where the matching score S _HSV and the matching score S _SEG are multiplied by 0.5 and then summed, but this ratio can be set as appropriate.

  As described above, the image collation apparatus 20 according to the first embodiment identifies the weight target area from the registered image data A and the input image data B, and the orientation of the person included in the registered image data A and the input image data B. The weight to be given to the weight target area is determined based on the combination of the registered image data A and the input image data B while applying the determined weight to the weight target area. For this reason, even when the orientations of the persons included in the registered image data A and the input image data B are different, image verification can be performed accurately and efficiently.

  In the first embodiment, the case where the weights are determined using the orientations of persons included in the registered image data and the input image data has been described. In the second embodiment, the weights are included in the registered image data and the input image data, respectively. A case where the weight is determined using the personal belongings will be described.

  FIG. 16 is an explanatory diagram for explaining the features of the image collating apparatus according to the second embodiment. Here, it is assumed that the registered image data A and the input image data B are collated using the color matching degree of the person's clothes and personal belongings.

  As shown in FIG. 16A, when the registered image data A3 includes an image of a person who possesses a heel and the input image data B4 includes an image of a person who possesses a heel, As indicated by the hatched area, the area corresponding to the ridge is used as a characteristic part to increase its weight. This is because if the belongings such as a bag are the same, there is a high possibility that they are the same person, and the matching accuracy can be improved by using the color as a characteristic part. In addition, not only a bag but other personal belongings, a hat, etc. can also be made into a characteristic part.

  On the other hand, as shown in FIG. 16B, when the registered image data A3 includes an image of a person with a heel and the input image data B5 includes an image of a person without a heel. As shown by the white area in the figure, the weight of the area corresponding to the eyelid is reduced. There is a possibility that items such as bags may be left in the middle, and if the items are not the same, it is not the basis for not being the same person. This is because there is a fear.

  In this way, by accurately changing the weight of the area corresponding to the possessed item according to the presence or absence of the possessed item of the person in the registered image data A and the input image data B, an accurate matching process can be performed by effectively using the characteristics of the possessed item. Can be performed.

  Next, the configuration of the image collating apparatus according to the second embodiment will be described. FIG. 17 is a functional block diagram illustrating the configuration of the image collating apparatus according to the second embodiment. As shown in FIG. 17, in the image matching device 50, the registration data 54 stored in the storage unit 21 is different from the registration data 21a shown in the first embodiment. The operations of the preprocessing unit 51, the weight target region specifying unit 52, and the weight determining unit 53 are different from those of the preprocessing unit 22a, the weight target region specifying unit 22b, and the weight determining unit 22c shown in the first embodiment. Since other configurations and operations are the same as those of the image collating apparatus 20 shown in the first embodiment, the same components are denoted by the same reference numerals and description thereof is omitted.

  The registered data 54 has a plurality of registered image data A, and each registered image data A is associated with personal belongings data. The plurality of registered image data A are generated for different persons, for example, and one registered image data A includes one person image. The personal belongings data indicates personal personal belongings in the corresponding registered image data A. FIG. 17 shows a state in which the belongings data A3b is associated with the registered image data A3.

  The preprocessing unit 51 is a processing unit that generates input image data B from imaged data captured by the camera 10 as preprocessing for collation. Specifically, the pre-processing unit 51 detects a human image from the imaging data, estimates the presence or absence of personal belongings, and cuts out a region including the human image as a human region. Then, the size of the cut-out human region is normalized, and the input image data B is generated by flattening the pixel histogram. It should be noted that any method can be used to detect images of persons and personal belongings. For example, it is possible to extract an image of a person by cutting out an image including both a person and personal belongings from the difference between captured image data that have been continuously captured and performing pattern matching of the person on this image. If an image of a person is excluded from an image including both the person and the belongings, an image of the belongings can be obtained.

  The weight target area specifying unit 52 performs processing for specifying, as a weight target area, an area corresponding to a possessed item when there is a possessed item with respect to a person image included in the registered image data A and the input image data B.

  The weight determination unit 53 determines the weight to be given to the weight target area based on the correspondence relationship between the registered image data A and the input image data B. Here, the weight determination unit 53 uses personal belongings for the correspondence between the registered image data A and the input image data B. Specifically, the weight determination unit 53 estimates that a person included in the input image data B has a belonging and indicates that the person has the possession based on the possession data associated with the registered image data A. If so, the weight given to the weight target area is increased. On the other hand, when there is no possession in one of the input image data B or the registered image data A, the weight to be given to the weight target area is lowered.

  Next, the processing procedure of the image collation apparatus 50 will be described. FIG. 18 is a flowchart showing the processing procedure of the image collating apparatus 50. First, when imaging data is received from the camera 10 (step S101), the preprocessing unit 51 of the image matching device 50 performs preprocessing on the imaging data to generate input image data B (step S602). By this preprocessing, the presence / absence of belongings in the input image data B is estimated.

  Further, the determination processing unit 22e selects one of the plurality of registered image data A (step S103). The weight target area specifying unit 52 and the weight determining unit 53 perform a weight setting process using the input image data B generated in step S602 and the registered image data A selected in step S103 (step S604). In this weight setting, an area corresponding to the possessed item is specified as a weight target region, and the weight is determined by a combination of presence / absence of the possessed item in the input image data B and the registered image data A.

The total score calculation unit 22d collates the input image data B and the registered image data A while applying the weight set in step S104 to the weight target region, and calculates a collation score S _HSV (step S605). Here, the input image data B and the registered image data A are divided into a weight target region corresponding to the possessed item and a person region other than the possessed item, and the total score calculation unit 22d performs collation for each region and performs a collation score. S _HSV is calculated. The subsequent processing is the same as that in the flowchart shown in FIG.

  As described above, the image collation apparatus 50 according to the second embodiment specifies the weight target area corresponding to the belongings from the registered image data A and the input image data B, and the registered image data A and the input image data B are identified. The weight to be given to the weight target area is determined based on the combination of presence / absence of personal belongings included, and the registered image data A and the input image data B are collated while applying the determined weight to the weight target area. . Therefore, image verification can be performed accurately and efficiently regardless of the presence or absence of personal belongings included in the registered image data A and the input image data B.

  In the first and second embodiments described above, the verification of the person has been described as an example. However, the present invention is not limited to the verification of the person, and the case where other objects such as a vehicle are compared is also described. Can be applied.

  In the first and second embodiments, the case where the matching process is performed using the color matching degree on the appearance of the person is shown. However, when the performance of the monitoring camera or the like is improved and high-resolution image data can be acquired. For example, it is also possible to perform a collation process based on a pattern of clothes or a shape feature depending on an attachment. In the present invention, the input image data and the registered image data are collated by weighting according to the mode of both images, and the collation processing method is not particular.

  In the second embodiment, the case where the belongings are used as the characteristic part of the target object has been described as an example. Any feature can be used if it is relatively easy.

  Each configuration illustrated in the present embodiment is functionally schematic and does not necessarily need to be physically configured as illustrated. In other words, the form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof may be functionally / physically distributed / integrated in arbitrary units according to various loads and usage conditions. Can be configured.

  The image collation apparatus and the image collation method according to the present invention are suitable for accurately and efficiently performing image collation when the input image and the registration object include different objects.

DESCRIPTION OF SYMBOLS 10 Camera 20 Image collation apparatus 21 Memory | storage part 21a, 54 Registration data 21b Candidate image list 22 Control part 22a, 51 Pre-processing part 22b, 52 Weight object area | region specific | specification part 22c, 53 Weight determination part 23d Total score calculation part 22e Determination processing part 22f Display control unit 30 Operation unit 40 Display unit A, A1 to A3 Registered image data A1a Orientation data A1b Inventory data B, B1 to B5 Input image data

Claims (7)

An image collation device that collates input image data in which a predetermined object is reflected and registered image data registered in advance,
A weight target area specifying unit that divides a target area of the input image data and / or the registered image data and specifies a weight target area that forms a part of the divided area;
A weight determining unit that determines a weight to be given to the weight target region based on a correspondence relationship between the input image data and the registered image data;
An image collation apparatus comprising: a collation processing unit that collates the input image data and the registered image data while applying the weight determined by the weight determination unit to the weight target region. The weight determination unit
When the aspect of the object included in the input image data and the aspect of the object included in the registered image data match or are similar, the weight given to the weight target area is increased, and the input image data The image verification according to claim 1, wherein when the aspect of the object included is not similar to the aspect of the object included in the registered image data, the weight assigned to the weight target area is reduced. apparatus. The weight determination unit
When the direction of the object included in the input image data matches the direction of the object included in the registered image data, the weight to be given to the weight target area is increased and included in the input image data The image collating apparatus according to claim 2, wherein when the direction of the object does not match the direction of the object included in the registered image data, the weight to be given to the weight target area is reduced. The weight determination unit
When the feature part exists in the target object included in the input image data and the feature part exists in the target object included in the registered image data, the weight to be given to the weight target region is increased. The image collating apparatus according to claim 2, wherein when one of the input image data and the registered image data does not have a characteristic portion, a weight to be given to the weight target region is lowered. The collation processing unit
While applying the weight determined by the weight determination unit to the weight target area, the weighted histograms of the input image data and the registered image data are calculated, and the input image data is based on the calculated weighted histograms. The image collating apparatus according to claim 3, wherein collation processing is performed on the registered image data. The collation processing unit
The input image data and the registered image data are each divided into small regions having similar colors or shades, and the weights determined by the weight determination unit are applied to the small regions corresponding to the weight target regions. The image collating apparatus according to claim 3, wherein a collation process is performed between the small area of the input image data and the small area of the registered image data. An image collation method in an image collation apparatus for collating input image data in which a predetermined object is reflected and registered image data registered in advance,
A weight target area specifying step of dividing a target area of the input image data and / or the registered image data and specifying a weight target area forming a part of the divided area;
A weight determination step for determining a weight to be given to the weight target region based on a correspondence relationship between the input image data and the registered image data;
An image collation method comprising: a collation processing step of collating the input image data and the registered image data while applying the weight determined by the weight determination unit to the weight target region.

Images