JP2013210844A - Image collation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem in which image collation for collating whether object images in two images are the same object is affected by a variation in appearance due to an illumination condition difference between the images.SOLUTION: A color conversion unit 212 calculates cumulative histograms for a color luminance value of a pixel in an object image of interest of two images to be collated, and subjects one object image to luminance conversion for making its cumulative histogram similar to the cumulative histogram of the other object image. An inter-person similarity calculation unit 213 collates the one object image subjected to the luminance conversion with the other object image.

Description

  The present invention relates to an image collation apparatus that collates whether object images captured in respective images are of the same target object.

  For the purpose of following the footsteps of a suspicious person, it may be required to search for an image of the same person from images captured by surveillance cameras installed in different places. For example, in order to specify when a suspicious person confirmed in the video on the second floor of the building has entered the building, it is required to search for the video at the entrance of the building.

  According to the image processing technique for tracking a moving object, it is possible to specify an image in which the same person is shown using the continuity of person positions. However, surveillance cameras are often installed discretely for reasons such as installation costs, and it is difficult to use the continuity of person positions between remote surveillance cameras. For this reason, images of a person tracked by different surveillance cameras are collated with colors such as clothes and color arrangement characteristics.

  In this way, when collating humans imaged by different surveillance cameras, color misregistration caused by a difference in illumination conditions causes a mismatch between the persons.

  Therefore, conventionally, a common subject is photographed by each camera and the color distribution of the subject image is matched between the cameras to correct differences in illumination conditions (Patent Document 1), or a large number of people are imaged by each camera. Thus, the difference in lighting conditions is corrected by matching the average color characteristics of human images between cameras (Non-patent Document 1).

JP 2007-19713 A

OmarJaved, Khurram Shafique, Zeeshan Rasheed, Mubarak Shah, “Modeling inter-camera space-time andappearance relationships for tracking across non-overlapping views”, Computer Vision and Image Understanding, Vol. 109, Issue 2, pp. 146-162, Feb , 2008.

  However, the illumination conditions vary depending on the location even within the field of view of the same surveillance camera (for example, the sun and shade in the field of view) and change from moment to moment. For this reason, even if the difference in illumination condition is corrected in advance for each camera as in the prior art, color misregistration occurs depending on the person position and imaging time, and the matching accuracy is not sufficiently improved.

  Also, since individual clothes are diverse, even if the difference in lighting conditions is corrected in advance with the color characteristics of a specific person or the average color characteristics of many persons as in the prior art, it was not used for correction A color shift occurred for a person wearing colored clothes, and the matching accuracy was not sufficiently improved. Furthermore, it has been difficult to collect images of a large number of people in advance for all surveillance cameras in terms of labor and cost.

  The problem of these illumination conditions occurs not only when the objects captured by different monitoring cameras are collated, but also when an object once deviated from the field of view of the monitoring camera reappears in the field of view of the same monitoring camera.

  The present invention has been made in view of the above problems, and corrects illumination conditions that change according to location and time while avoiding a decrease in matching accuracy due to color misregistration due to differences in color characteristics among individuals. An object of the present invention is to provide an image collation apparatus capable of collating an object with high accuracy.

  The image collation device according to the present invention collates whether or not the target object images captured in each of the two images are of the same target object, and accumulates luminance values in the target object images of the respective images. A luminance conversion unit that calculates a histogram and applies luminance conversion to the one target object image for luminance conversion that approximates the cumulative histogram of the one target object image to the cumulative histogram of the other target object image; And a collation unit that collates the one target object image with the other target object image.

  In another image collation device according to the present invention, the luminance conversion unit performs the luminance conversion for approximating a cumulative color histogram, which is the cumulative histogram for the luminance value for each color component, between the target object images.

  In still another image collation device according to the present invention, the luminance conversion unit divides the target object image in the two images into a plurality of partial regions, and the partial regions corresponding to each other of the target object images. The luminance conversion is performed every time.

  In another image collation device according to the present invention, the two object images of interest are object image groups each consisting of a plurality of object images determined to be of the same target object, and the luminance conversion unit The cumulative histogram in each object image group is calculated.

  In still another image collation device according to the present invention, the two object images are the same target object and an object image group consisting of a plurality of object images determined to have the same shape characteristics. The luminance conversion unit calculates the cumulative histogram in each object image group.

  According to the present invention, in the case of images of the same object, the difference in illumination conditions is corrected without color shift by luminance conversion. However, in the case of an image of a different object, color shift due to individual difference occurs due to luminance conversion. The difference between the collation result when the object is the same and the collation result when the object is different opens. Therefore, collation accuracy can be improved while correcting the illumination conditions.

It is a block diagram which shows the structure of the outline of the person information generation apparatus which comprises the person identification / search system which is embodiment of this invention. It is a block diagram which shows the structure of the outline of the person identification apparatus which comprises the person identification and search system which is embodiment of this invention. It is explanatory drawing which shows an example of color conversion when the person of a query and the person of a database are the same persons. It is explanatory drawing which shows an example of the color conversion when the person of a query and the person of a database are others. It is a schematic flowchart explaining operation | movement of a person information generation apparatus. It is a schematic flowchart explaining operation | movement of a person identification apparatus.

  Hereinafter, a person identification / search system according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The person identification / retrieval system has a function of checking whether a person shown in two images is the same by a person image that is a target object image in the image, with the person as a target object. The person identification / retrieval system includes a person information generation device 1 and a person identification device 2. The person information generating device 1 generates image information (person information) of a person appearing in each imaging area for each person from a plurality of frames of images (time-series images) obtained by imaging different imaging areas by a plurality of monitoring cameras. This person information includes a person image separated from the background. The person identification device 2 compares person images with the person information generated by the person information generation device 1 to detect person information of the same person. The person identification / retrieval system enables a cross-sectional search of a person who moves across a plurality of imaging regions by collating the person images. Hereinafter, the person information generation apparatus 1 and the person identification apparatus 2 will be described.

[Configuration of Person Information Generating Device 1]
FIG. 1 is a block diagram showing a schematic configuration of the person information generating apparatus 1. The person information generation apparatus 1 includes an input unit 10, a signal processing unit 11, and a storage unit 12.

  The input unit 10 is a video recording device such as a digital video recorder (DVR) that records a plurality of monitoring cameras that output time-series images or time-series images that have been captured by each monitoring camera, and is connected to the signal processing unit 11. The

The signal processing unit 11 is a CPU (Central
It is configured using an arithmetic device such as a processing unit (DSP), a digital signal processor (DSP), or a micro control unit (MCU), and is connected to the input unit 10 and the storage unit 12. The signal processing unit 11 reads out and executes a program from the storage unit 12, and functions as a person tracking unit 110, a person feature extraction unit 111, a similar posture image group generation unit 112, a person information generation unit 113, and the like.

  The person tracking unit 110 acquires a time series image for each monitoring camera from the input unit 10, generates a time series (time series person image) of person images of each person appearing on the time series image by the person tracking process, The data is output to the person feature extraction unit 111. Specifically, the person tracking unit 110 extracts a person image from each frame of the time series image by background difference processing or the like, and generates a time series person image by associating the person image of the same person between the frames by pattern matching or the like. To do. The data of each person image is transferred by transferring a frame number for identifying a frame in which the person image exists and a binary mask image representing a person area in the frame.

  The person feature extraction unit 111 extracts appearance feature values and shape feature values from each person image generated by the person tracking unit 110, and outputs the extracted feature values to the similar posture image group generation unit 112 and the person information generation unit 113. To do.

  The appearance feature amount is an apparent feature amount of a person, and specifically, a luminance feature amount of a person image. The person feature extraction unit 111 extracts an RGB color histogram of each person image as an appearance feature value. This is a color feature amount that characterizes a person with colors such as clothes and skin. An HSV color histogram may be extracted instead of the RGB color histogram. Alternatively, the person feature extraction unit 111 extracts a luminance gradient histogram of each person image as the appearance feature value. This is a texture feature amount that characterizes a person by a clothing pattern or the like. As the appearance feature amount, both the color feature amount and the texture feature amount of each person image can be extracted, and various other luminance feature amounts can be added.

  The shape feature amount is a feature amount that characterizes the posture and orientation of a person, and for example, a shape context or an edge direction histogram can be used.

  The similar posture image group generation unit 112 divides each time-series human image generated by the person tracking unit 110 into similar posture image groups composed of person images having similar shape feature amounts extracted by the person feature extraction unit 111, and generates groups. Is output to the person information generation unit 113.

  First, the similar posture image group generation unit 112 obtains the similarity of the shape feature amount of the person image between frames in each time-series person image. The similarity of the shape feature amount can be obtained by using a matching method such as a dynamic programming method. Next, the similar posture image group generation unit 112 applies an existing clustering method such as a mean shift (MeanShift) method or a quick shift (QuickShift) method to the obtained similarity to cluster the human images of each time-series human image. By doing so, a similar posture image group is generated.

  The person information generation unit 113 calculates, for each similar posture image group generated by the similar posture image group generation unit 112, a representative appearance feature amount that represents the appearance feature amount of the person image constituting the group, and the mask image and the frame A person image is cut out based on the number and the time series image, and for each time series person image, the person ID, the frame number list of each similar posture image group, the person image and shape feature amount for each frame number, and the representative of each similar posture image group The personal information 120 associated with the appearance feature value is stored in the storage unit 12.

  The person information generation unit 113 calculates the representative appearance feature value by averaging the appearance feature values of the person images in each similar posture image group. The average number of person images may be used, but a specified number of person images close to the average value may be used. Alternatively, the appearance feature value closest to the average appearance feature value of the person images of each similar posture image group may be used as the representative appearance feature amount, or the clustering result in the similar posture image group generation unit 112 may be used as the representative feature value. The appearance feature amount corresponding to the cluster center of each similar posture image group can be referred to as the representative appearance feature amount. Further, the person information generation unit 113 may generate the representative shape feature amount of each similar posture image group in the same manner as the appearance feature amount with respect to the shape feature amount.

  The storage unit 12 includes, for example, a storage device such as a hard disk drive (HDD), a RAM (Random Access Memory), etc., and stores the personal information 120 generated by the personal information generation unit 113 as well as temporary calculation data. It is used by the signal processing unit 11 as a storage area (not shown). In addition, the storage unit 12 stores various programs and various data used in the signal processing unit 11, and inputs / outputs such information to / from the signal processing unit 11.

[Configuration of Person Identification Device 2]
FIG. 2 is a block diagram showing a schematic configuration of the person identification device 2. The person identification device 2 includes an input unit 20, a signal processing unit 21, a storage unit 22, and an output unit 23. As already described, the person identification device 2 performs an identification process for finding the same person using the person information 120 created by the person information generation device 1. In the present embodiment, a user requests a search for a person appearing in a time-series image of a certain monitoring camera (hereinafter referred to as camera A), and the person identification device 2 receives another monitoring camera (hereinafter referred to as camera B). It is assumed that the person is searched from the time-series images photographed in (1).

  The input unit 20 is an input device such as a keyboard and a mouse, and is connected to the signal processing unit 21 and is used when the user selects a person who wants to search or designates a camera B to be searched.

  Similar to the signal processing unit 11, the signal processing unit 21 is configured using an arithmetic device such as a CPU, and is connected to the input unit 20, the storage unit 22, and the output unit 23. The signal processing unit 21 reads out and executes the program from the storage unit 22 and serves as a person information acquisition unit 210, a similar posture image group selection unit 211, a color conversion unit 212, a person similarity calculation unit 213, a collation result generation unit 214, and the like. Function.

  The person information acquisition unit 210 reads the person information (person information 220 </ b> A) of the person requested to be searched from the input unit 20 (hereinafter referred to as a query person) from the storage unit 22. In addition, the person information acquisition unit 210 receives from the storage unit 22 person information groups (person information 220B) of a plurality of persons (hereinafter referred to as DB (database) persons) designated by the monitoring unit as a collation partner from the input unit 20. Read. Then, the person information acquisition unit 210 outputs the person information 220 (220A, 220B) of the person specified as the search target or collation partner to the similar posture image group selection unit 211.

  The similar posture image group selection unit 211 has a similar posture image group having the most similar posture from the similar posture image group stored in the person information of the query person and the similar posture image group stored in the person information of each DB person. Select a pair. The similar posture image group selection unit 211 outputs personal information to which the information for specifying each selected similar posture image group is added to the color conversion unit 212.

  The similar posture image group pair is selected by comparing the similarity between the shape feature amounts of the similar posture image groups. Similarity between similar posture image groups is determined by using a matching method such as dynamic programming in the same manner as the method used by the similar posture image group generation unit 112 of the person information generation device 1. Can be obtained by comparing.

  The query person and the DB person are collated by collating the selected pair of person images. By selecting and comparing human images with similar postures in this way, it is possible to suppress a decrease in collation accuracy due to differences in human postures and orientations.

  The color conversion unit 212 uses the color between the similar posture image group pairs selected by the similar posture image group selection unit 211 to absorb the difference in the lighting conditions between the person image of the query person and the person image of the DB person. Perform the conversion process. The color conversion process is performed by BTF (Brightness Transfer Function).

Specifically, the color conversion unit 212 calculates a cumulative color histogram of each person image of the query person and DB person, converts a luminance value (color luminance value) of one color component of the cumulative color histogram, and approximates the other. Each color luminance value is converted so that That is, one cumulative color histogram is a function H ₁ (B ₁ ) for the color luminance value B ₁ , the other cumulative color histogram is a function H ₂ (B ₂ ) for the color luminance value B ₂ , and a color luminance value B ₁ by BTF. Is expressed as f (B ₁ ), the color conversion unit 212 defines the function f so that H ₁ (B ₁ ) matches H ₂ (f (B ₁ )). Cumulative frequency _{H 1} of the color conversion section 212 and the cumulative frequency _{H 2 (B 2)} to the luminance values 0 to B ₂ for each luminance value _{B 2,} until the luminance values 0 to B ₁ for each luminance value _{B 1} ( B ₁ ) is obtained, and a conversion f (B ₁ ) is performed in which each of the luminance values B ₁ is replaced with a luminance value B ₂ that satisfies H ₁ (B ₁ ) = H ₂ (B ₂ ). This conversion is performed for each color component. In practice, since the luminance value and frequency are discrete, in general, conversion that completely satisfies H ₁ (B ₁ ) = H ₂ (B ₂ ) cannot be obtained, and the color conversion uses one accumulated color histogram for the other. The conversion approximates the color histogram. Here, the change characteristic represented by the curve (broken line) of the cumulative histogram is called an integral characteristic, and approximating the characteristic by BTF is expressed as integrating the integral characteristic.

  Since the color conversion performed by the color conversion unit 212 is performed on a human image excluding the background, the difference in the background color distribution does not affect the color conversion. The color conversion performed by the color conversion unit 212 is conversion that matches the color characteristics of the person image of the specific person, and is conversion that imposes the constraint condition that does not change the order of the luminance values as described above. For this reason, even if another person with different original color distributions tries to match the integral characteristics, the color distribution cannot be made due to the constraint conditions and the color distributions are not similar. However, if the individuals who have the same original color distribution are trying to match the integral characteristics, even if there are constraints, they can be matched without causing color shifts, and the lighting conditions are corrected and both color distributions are It will be similar.

  As a result, it is possible to widen the difference between the collation between the principals and the collation between others, and the collation accuracy can be improved.

  FIG. 3 is an explanatory diagram illustrating an example of color conversion when the query person and the DB person are the same person, and FIG. 4 is an explanatory diagram illustrating an example of color conversion when both persons are other persons.

  The upper half of FIGS. 3 and 4 shows a cumulative color histogram in which the relative frequency of the luminance value for each color of RGB in the pixel of the person image is accumulated. The left side is the cumulative color histogram of the query person, and the right side is the cumulative color histogram of the DB person. These cumulative color histograms are before color conversion. On the other hand, the lower half of FIGS. 3 and 4 shows a color histogram after the color luminance value of the query person is converted and the integral characteristics of the query person and the DB person are combined. The left side is a query person, and the right side is a DB person. 3 and 4, the dotted line corresponds to red (R), the alternate long and short dash line corresponds to green (G), and the solid line corresponds to blue (B).

  In the example shown in FIG. 3, the query person is the same as the DB person, but the lighting conditions are different. Specifically, the query person is taken in a state where the illumination is dark, and the case where the difference in color in the person image is relatively unclear is taken as an example. Therefore, the difference in the integral characteristics of each color in the left cumulative color histogram is smaller than that in the right cumulative color histogram. In this case, if the color conversion is performed so that the integral characteristic of the query person matches the DB person, for example, the color histogram of each color of the query person has a peak position and a relative size similar to those of the DB person. That is, a similar color histogram is obtained. In the person image of the query person represented by the converted color luminance value, only the illumination condition is corrected well, and the skin of the person image and the color of the clothes become similar to the person image of the DB person.

  On the other hand, in the example shown in FIG. 4, the color of the clothes and the degree of skin exposure of the query person and the DB person are different. Therefore, for example, the order of RGB differs with respect to the speed of increase of the integral characteristics. In this case, if the color conversion is performed so that the integral characteristics of the query person and the DB person are matched, the relative positional relationship and relative strength between the peaks of each color in the color histogram of the query person are the original characteristics of the query person. Change from color histogram. Therefore, the person image of the query person after conversion has a distorted color distribution, and the illumination condition does not match the condition in the person image of the DB person.

  The color conversion described above opens a difference in the degree of color matching between the person and the other person, so that the person verification accuracy can be improved while correcting the illumination conditions.

  In addition, by performing this color conversion for each of a plurality of color components, it is possible to maintain the above-mentioned difference, for example, even if one color component is accidentally matched by another person, the other color component cannot be completely matched. be able to.

  Since this color conversion is adapted to the color distribution unique to the person image of the specific person, the above difference can be widened.

  Further, since this color conversion is performed on a human image excluding the background, the effect of widening the above difference can be enhanced.

  Furthermore, since this color conversion is performed using an average cumulative histogram of time-series human images, even if some background pixels are mixed due to human image extraction errors, the influence can be diluted, so the effect of widening the above difference is enhanced. be able to.

  In addition, since this color conversion is performed using an average cumulative histogram of time-series human images with similar shape features, it is possible to suppress the slowdown of personality due to changes in posture and orientation, and to increase the effect of widening the above difference. it can.

  Note that, when performing color conversion of a similar posture image group, the accumulated color histogram is obtained by summing up all the human image pixels in the group. Here, the size of the person image in the group may have a difference according to the relative positional relationship between the person and the camera in the real world. It is preferable to consider the difference in the size of the person image when the histogram is tabulated. For example, each frequency value of the accumulated color histogram of each person image is normalized by dividing by the number of pixels constituting the person image, and the normalized accumulated color histogram of each person image is averaged within the similar posture image group. Used for color conversion.

  In the above color conversion, the BTF is constructed from the accumulated color histogram of the entire person image. However, the person image is divided into a plurality of partial areas, and the BTF is constructed from the accumulated color histogram in the corresponding partial areas of the person images. You can also. For example, it is possible to divide a human image into an upper body region and a lower body region and construct a BTF from the upper body regions and the lower body regions from the respective regions. In this case, for example, when color conversion is performed between a person whose upper body is blue and lower body is brown and a person whose upper body is brown and lower body is blue, color conversion is performed using a BTF constructed for each divided region. By performing the above, it is possible to widen the difference in the degree of color matching between the person himself and another person, and the collation accuracy can be improved.

  The inter-person similarity calculation unit 213 calculates the similarity (inter-person similarity) between the person image of the query person that has been subjected to color conversion by the color conversion unit 212 and the person image of each DB person. The person-to-person similarity is output to the matching result generation unit 214.

  The similarity between persons is calculated as the similarity of appearance feature values. Specifically, the inter-person similarity calculation unit 213 re-extracts the representative appearance feature amount from the color-converted person image constituting the similar posture image group of the query person that is one of the similar posture image group pairs, and The representative appearance feature amount of the object image constituting the similar posture image group of each DB person who is the other of the similar posture image group pair is read from the person information 220B, and the representative appearance feature amount of the query person and the representative appearance feature of each DB person are read. The similarity with the quantity is calculated as the similarity between persons. The similarity of the RGB histogram, which is an appearance feature value, can be obtained by calculating a histogram intersection. Even when an HSV histogram or a luminance gradient histogram is used instead of the RGB histogram, these histogram intersections may be calculated. Further, when a plurality of types of feature quantities are used as appearance feature quantities, the similarity between the feature quantities can be weighted and added with a predetermined weight to obtain the similarity between persons.

  The collation result generation unit 214 determines a person whose DB similarity calculated by the person similarity calculation unit 213 is equal to or greater than a predetermined identification threshold as the same person as the query person. The person information 220 of the person is output to the output unit 23. On the other hand, if there is no DB person having a similarity between persons equal to or higher than the identification threshold, it is determined that there is no corresponding person and a message to that effect is output to the output unit 23.

  Similar to the storage unit 12, the storage unit 22 includes a storage device such as an HDD, and stores personal information 220 </ b> A and 220 </ b> B and is used by the signal processing unit 21 as a temporary storage area (not shown) for calculation. In addition, the storage unit 22 stores various programs and various data used in the signal processing unit 21, and inputs / outputs such information to / from the signal processing unit 21.

  Here, the person information 220A stores person information of a plurality of persons generated from the time-series images of the camera A by the person information generation apparatus 1, and the person information 220B stores the person information generation apparatus 1 in time series of the camera B. The personal information of a plurality of persons generated from the image is stored. In this embodiment, the case where there are two cameras has been described. However, when there are more than two cameras, the same number of person information 220 as the number of cameras is stored in the storage unit 22.

  The output unit 23 includes a display. The output unit 23 is connected to the signal processing unit 21 and outputs the result generated by the matching result generation unit 214.

  When the person information generation device 1 and the person identification device 2 are configured as an integrated person identification / search system, the signal processing unit 11 and the signal processing unit 21 can be configured by a common arithmetic device, and the storage unit 12 and the storage unit 22 can also be constituted by a common storage device.

[Operation of Person Information Generating Device 1]
FIG. 5 is a schematic flowchart for explaining the operation of the person information generating apparatus 1.

  The person tracking unit 110 performs background difference processing on each image of the time-series image input from the input unit 10 to extract a person image by removing background pixels from the image, and generates a mask image ( Step S10). Then, the person tracking unit 110 performs template matching between the person image of the current frame extracted in step S10 and the person images extracted up to the previous frame to associate person images of the same person with each other (step S11). The person tracking unit 110 assigns a common person ID to the person image of the same person, and assigns a new person ID to the person image of the new person. The person tracking unit 110 adds the frame number of the current frame, the mask image of the person image extracted in the current frame, and the person ID to the storage unit 12 in association with each other.

  The person feature extraction unit 111 extracts a shape feature amount from the person image extracted by the person tracking unit 110 in step S10, and adds it to the storage unit 12 in association with the person image (step S12). Also, the person feature extraction unit 111 extracts appearance feature values from the person image, and adds the appearance feature amount to the storage unit 12 in association with the person image (step S13).

  If there is a person who has finished tracking (YES in step S14), the process proceeds to step S15. If not (NO in step S14), the process in steps S10 to S13 is performed on a newly input frame. repeat. The person feature extracting unit 111 collects the image information of the tracking-completed person recorded in steps S <b> 10 to S <b> 13 and outputs it to the similar posture image group generation unit 112 and the person information generation unit 113. The similar posture image group generation unit 112 generates a similar posture image group by clustering the time-series human images of the tracking-completed persons based on the shape feature amount extracted in step S12, and generates a frame number list for each group as the person information. It outputs to the production | generation part 113 (step S15).

  The person information generation unit 113 generates a representative appearance feature amount by averaging the appearance feature amounts of the person images constituting the group for each similar posture image group of the tracking-finished person generated in step S15 (step S15). S16). In step S15, the person information generation unit 113 uses the person ID of the tracking end person obtained in steps S10 to S12, the time-series person image of the person and each frame number, and the shape feature amount of each person image in step S15. The person information 120 of the tracking end person is generated by adding the information of the generated similar posture image group and the representative appearance feature value generated in step S16, and is stored in the storage unit 12 (step S17). When the person information 120 of the tracking end person is generated, the signal processing unit 11 returns the process to step S10 and proceeds to the process of the next frame.

[Operation of Person Identification Device 2]
FIG. 6 is a schematic flowchart for explaining the operation of the person identification device 2.

  The user operates the input unit 20 to select one of the persons shown in the time-series image of the camera A as a query person (step S20), and selects the camera B as a camera to be associated ( Step S21). The selection of the camera B is equivalent to selecting all persons shown in the time-series image of the camera as DB persons.

  The person information acquisition unit 210 reads the person information 220A of the query person specified in step S20 from the storage unit 22 (step S22).

  In the subsequent loop processing of steps S23 to S28, the person similarity between the query person and all DB persons is calculated.

  The person information acquisition unit 210 sequentially reads the person information 220B of each DB person from the storage unit 22 (step S23). Then, the similar posture image group selection unit 211 compares the shape feature amount of each similar posture image group of the query person with the shape feature amount of each similar posture image group of the DB person read in step S23, and the shape feature amount. A pair of similar posture image groups with the most similar is selected (step S24).

  The color conversion unit 212 calculates a cumulative color histogram of each of the similar posture image group of the query person and the similar posture image group of the DB person constituting the pair selected in step S24 (step S25), and the similar posture image of the query person. A color conversion process for approximating the accumulated color histogram of the group to the accumulated color histogram of the similar posture image group of the DB person is performed on each person image constituting the similar posture image group of the query person (step S26).

  The person-to-person similarity calculation unit 213 extracts a representative appearance feature value from the similar posture image group of the query person that has undergone color conversion in step S26, and the representative appearance feature value and the similar posture image group selected for the DB person. The similarity with the representative appearance feature value is calculated as the similarity between the query person and the DB person (step S27).

  The person similarity calculation unit 213 checks whether or not the person similarity is calculated for all DB persons specified in step S21 (step S28). If not calculated for all (NO determination in step S28), the processes in steps S23 to S27 are repeated for the uncalculated DB person. On the other hand, if the inter-person similarity is calculated for all DB persons (YES in step S28), the process proceeds to step S29.

  The matching result generation unit 214 selects the maximum person similarity from among the person similarities calculated for each DB person by the loop processing in steps S23 to S28, compares the person similarity with the identification threshold, and determines the maximum person similarity. Is equal to or greater than the identification threshold (YES in step S29), the DB person whose inter-person similarity is calculated is determined to be the same person as the query person, and the person information of these persons is output to the output unit 23. Output (step S30).

  If the maximum similarity between persons is less than the identification threshold (NO determination in step S29), the matching result generation unit 214 outputs to the output unit 23 that the DB person corresponding to the query person has not been detected. S30 is omitted.

  The output unit 23 outputs the person information of the same person or the information of no corresponding person input from the collation result generation unit 214 to the display or the like as the identification process result (step S31), and ends the process.

[Modification]
(1) In step S26, an example of color conversion in which the color conversion unit 212 approximates the accumulated color histogram of the query person's image to the accumulated color histogram of the DB person's image has been shown. Color conversion may be performed to approximate the color histogram to the cumulative color histogram of the query person's image.

  (2) In the above embodiment, a plurality of object images obtained by processing to track the same person as a target object image to be collated as to whether or not they are the same target object, that is, a person image of a query person and a person image of a DB person. The collation accuracy is improved by using groups with similar postures. On the other hand, a simple configuration in which a plurality of person images obtained by the process of tracking the same person is directly subjected to collation without being grouped by posture is also possible. Further, instead of a person image obtained from a plurality of frames, a person image obtained from a single frame may be used as a collation target.

  (3) In the above embodiment, the luminance conversion is performed on the color luminance value. However, for example, in a monochrome image, the luminance conversion may be performed on the luminance value Y not related to the hue.

  (4) The appearance feature value may be obtained from, for example, a person area divided into an upper body area and a lower body area. As a method of dividing the person area, for example, the mask image obtained by the person tracking unit 110 is divided into two in the height direction, or the position of the pixel corresponding to the person area of the mask image is directed in the height direction of the person. A method of obtaining the main axis and dividing it at equal intervals along the main axis is applicable. For example, the shape of a person appearing in the mask image can be approximated by an ellipse centered on the average coordinate, and the major axis of the ellipse can be used as the main axis.

  In this configuration, the collation accuracy can be increased by integrally determining the collation results of the respective areas. In the integrated determination, for example, a part of a plurality of divided areas or all of them are associated with the same person because the similarity with respect to the appearance feature amount is equal to or greater than a threshold value. Can do.

  DESCRIPTION OF SYMBOLS 1 Person information generation apparatus, 2 Person identification apparatus, 10,20 Input part, 11,21 Signal processing part, 12,22 Storage part, 23 Output part, 110 Person tracking part, 111 Person feature extraction part, 112 Similar posture image group Generation unit, 113 person information generation unit, 210 person information acquisition unit, 211 similar posture image group selection unit, 212 color conversion unit, 213 interperson similarity calculation unit, 214 collation result generation unit

Claims (5)

An image collation device that collates whether target object images captured in two images are of the same target object,
A cumulative histogram of luminance values in the target object image of each image is calculated, and luminance conversion is performed to approximate the cumulative histogram of one target object image to the cumulative histogram of the other target object image. A luminance conversion unit applied to the image;
A collation unit that collates the one target object image subjected to the luminance conversion and the other target object image;
An image collating apparatus comprising: The image collation apparatus according to claim 1,
The luminance conversion unit performs the luminance conversion for approximating a cumulative color histogram, which is the cumulative histogram of luminance values for each color component, between the target object images. In the image collation device according to claim 1 or 2,
The luminance conversion unit divides the target object image in the two images into a plurality of partial regions, and performs the luminance conversion for each of the partial regions corresponding to the two target object images;
An image matching apparatus characterized by the above. In the image collation device according to any one of claims 1 to 3,
The two object images of interest are object image groups each consisting of a plurality of object images determined to be of the same target object,
The luminance conversion unit calculates the cumulative histogram in each object image group;
An image matching apparatus characterized by the above. In the image collation device according to any one of claims 1 to 3,
The two object images of interest are an object image group consisting of a plurality of object images that are determined to have the same target object and the same shape characteristics,
The luminance conversion unit calculates the cumulative histogram in each object image group;
An image matching apparatus characterized by the above.

Images