JP2013210845A - Moving object collation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that, when object images including similar attitudes of a moving object are collected from a plurality of frames and image features are averaged, the image features are made dull by being affected with a variation in an illumination condition.SOLUTION: A similar attitude image group generating section 112 divides time-series object images into continuous frames in which image features are similar to each other. A person information generating section 113 calculates a representative feature which represents luminance features, constituting the continuous frames, of a moving object in the plurality of frames. A person identifying device 2 collates a plurality of time-series object images with each other regarding the representative features for each of the continuous frames and determines the time-series object images which are matched in the collation and include the continuous frames, as the same moving object.

Description

  The present invention relates to a moving object collation apparatus that collates whether time-series images each captured of a moving object are of the same moving object.

  For the purpose of following the footsteps of a suspicious person, it may be required to search for an image of the same person from images captured by surveillance cameras installed in different places. For example, in order to specify when a suspicious person confirmed in the video on the second floor of the building has entered the building, it is required to search for the video at the entrance of the building.

  According to the image processing technique for tracking a moving object, it is possible to detect a time-series image in which the same person is shown using the continuity of person positions. However, surveillance cameras are often installed discretely for reasons such as installation costs, and it is difficult to use the continuity of person positions between remote surveillance cameras. For this reason, time-series images of people tracked by different surveillance cameras are collated with appearance features such as clothes.

  When collating a person imaged by different surveillance cameras, the difference in lighting conditions depending on the location becomes a cause of identity mismatch or identity match. Such a problem of illumination conditions also occurs when a person who has once deviated from the field of view of the surveillance camera reappears in the field of view of the same surveillance camera. For this reason, conventionally, a position where a person is detected is stored in association with the detected person area image, and a person whose position and person area image are both similar is searched to limit candidates and prevent erroneous search. It was.

JP 2000-350187 A

  However, if the person detection position is used as a search condition, there are a large number of person images that cannot be candidates, and there is a problem that a sufficient search cannot be performed. In other words, the conventional technology has a problem that it is not possible to effectively use a plurality of frames of time-series images that are identified by tracking as those of the same person. In addition, matching between frames is likely to be an erroneous search factor because it is easily influenced by the influence of noise such as background images mixed due to errors in human region extraction and the fluctuation of the person caused by fluctuations in the posture and orientation of the person. It was.

  A process such as averaging a plurality of frames is suitable for suppressing the influence of noise and personal fluctuation. However, the lighting conditions vary depending on the time and place, and the direction and posture of the person change from moment to moment. Therefore, frames having different lighting conditions and postures and directions are mixed in the time-series image. For this reason, if a plurality of frames of a time-series image are inadvertently averaged, characteristics such as color are dulled, and collation accuracy is lowered.

  The present invention has been made in view of the above problems, and provides a moving object collation apparatus capable of collating a moving object with high accuracy by effectively using a time-series image that can include fluctuations in posture and orientation of a moving object and illumination fluctuations. The purpose is to provide.

  The moving object collation device according to the present invention is for collating whether or not a plurality of time-series object images each captured of a moving object are of the same moving object, the time-series object image being the moving object A continuous frame generation unit that divides the image features into continuous frames that are similar to each other, a representative feature calculation unit that calculates a representative feature that represents a luminance feature of the moving object in a plurality of frames constituting the continuous frame, and the plurality A collating unit that collates the time-series object images of the representative features for each successive frame and determines that the time-series object images including the consecutive frames that have been matched for matching are those of the same moving object. .

  In the moving object collation apparatus according to another aspect of the present invention, the representative feature calculation unit calculates the representative feature by averaging the luminance features of the moving object in a plurality of frames constituting the continuous frame.

  In the moving object matching device according to another aspect of the invention, the matching unit selects the continuous frames having the most similar shape characteristics of the moving object between the time-series object images, and matches the selected consecutive frames. .

  According to the present invention, the luminance characteristics of a moving object can be reduced while suppressing fluctuations in posture / direction and fluctuations in illumination by dividing and collating time-series images into continuous frames having the same orientation, attitude, and illumination conditions. Slowdown can be prevented and collation accuracy can be improved.

It is a block diagram which shows the structure of the outline of the person information generation apparatus which comprises the person identification / search system which is embodiment of this invention. It is a block diagram which shows the structure of the outline of the person identification apparatus which comprises the person identification and search system which is embodiment of this invention. It is a schematic flowchart explaining operation | movement of a person information generation apparatus. It is explanatory drawing which shows an example of a process of the similar attitude | position image group production | generation part. It is a schematic flowchart explaining operation | movement of a person identification apparatus.

  Hereinafter, a person identification / search system according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The person identification / retrieval system has a function of collating time-series object images over a plurality of frames of a moving object shown in an image with a person as a target object. The person identification / retrieval system includes a person information generation device 1 and a person identification device 2. The person information generating device 1 generates image information (person information) of a person appearing in each imaging area for each person from a plurality of frames of images (time-series images) obtained by imaging different imaging areas by a plurality of monitoring cameras. This person information includes a person image separated from the background. The person identification device 2 compares person images with the person information generated by the person information generation device 1 to detect person information of the same person. The person identification / retrieval system enables a cross-sectional search of a person who moves across a plurality of imaging regions by collating the person images. Hereinafter, the person information generation apparatus 1 and the person identification apparatus 2 will be described.

[Configuration of Person Information Generating Device 1]
FIG. 1 is a block diagram showing a schematic configuration of the person information generating apparatus 1. The person information generation apparatus 1 includes an input unit 10, a signal processing unit 11, and a storage unit 12.

  The input unit 10 is a video recording device such as a digital video recorder (DVR) that records a plurality of monitoring cameras that output time-series images or time-series images that have been captured by each monitoring camera, and is connected to the signal processing unit 11. The

  The signal processing unit 11 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and is connected to the input unit 10 and the storage unit 12. The signal processing unit 11 reads out and executes a program from the storage unit 12, and functions as a person tracking unit 110, a person feature extraction unit 111, a similar posture image group generation unit 112, a person information generation unit 113, and the like.

  The person tracking unit 110 acquires a time series image for each monitoring camera from the input unit 10, generates a time series (time series person image) of person images of each person appearing on the time series image by the person tracking process, The data is output to the person feature extraction unit 111. Specifically, the person tracking unit 110 extracts a person image from each frame of the time series image by background difference processing or the like, and generates a time series person image by associating the person image of the same person between the frames by pattern matching or the like. To do. The data of each person image is transferred by transferring a frame number for identifying a frame in which the person image exists and a binary mask image representing a person area in the frame.

  The person feature extraction unit 111 extracts appearance feature values and shape feature values from each person image generated by the person tracking unit 110, and outputs the extracted feature values to the similar posture image group generation unit 112 and the person information generation unit 113. To do.

  The appearance feature amount is an apparent feature amount of a person, and specifically, a luminance feature amount of a person image. The person feature extraction unit 111 extracts an RGB color histogram of each person image as an appearance feature value. This is a color feature amount that characterizes a person with colors such as clothes and skin. An HSV color histogram may be extracted instead of the RGB color histogram. Alternatively, the person feature extraction unit 111 extracts a luminance gradient histogram of each person image as the appearance feature value. This is a texture feature amount that characterizes a person by a clothing pattern or the like. As the appearance feature amount, both the color feature amount and the texture feature amount of each person image can be extracted, and various other luminance feature amounts can be added.

  The shape feature amount is a feature amount that characterizes the posture and orientation of a person, and uses a shape context. In another embodiment, an edge direction histogram can be used as the shape feature amount.

The similar posture image group generation unit 112 (continuous frame generation unit) is a similarity composed of human images of continuous frames having similar shape feature amounts extracted by the human feature extraction unit 111 from each time-series human image generated by the human tracking unit 110. Dividing into posture image groups, the group generation result is output to the person information generation unit 113. The similar posture image group generated by the grouping is a collection of human images having the same posture and orientation. Even if averaging is performed in groups with uniform postures and orientations, the image features of moving objects are less blunted due to changes in posture and orientation, so it is possible to suppress degradation in accuracy in collation using those image features. It becomes. In addition, time continuity is taken into consideration in the grouping. For example, when the continuity constraint time for similar posture image grouping is set to T _C (seconds), the time-series human image is divided at intervals of T _C seconds from the tracking start frame t _0, and each division time generated by the division is obtained. Group into similar posture images in the sequence image. The size of the continuous duty time T _C in the range of about 2 to 5 seconds for example, can be set according to the number of frames included in each divided time-series image. By making the time series continuity of the images included in the group in the group generation of similar posture images, even if the posture of the person is similar, images that are separated in time are classified into another group Even if the illumination variation occurs in the time-series image to be tracked, the size of the illumination variation within the group can be reduced. Therefore, even if averaging within the group is performed, there is little dullness due to illumination fluctuation of the image feature of the moving object, so that it is possible to suppress a decrease in accuracy in the collation using the obtained image feature.

  Hereinafter, a grouping method for similar posture images will be described.

  First, the similar posture image group generation unit 112 obtains the similarity of the shape feature amount of the person image between the temporally adjacent person images in each divided time series image. The similarity can be obtained by using a matching method such as a dynamic programming method. Next, the similar posture image group generation unit 112 applies a clustering technique such as an average shift (MeanShift) method or a quick shift (QuickShift) method to the obtained similarity to cluster the human images of each divided time-series image. To generate a similar posture image group.

  The person information generation unit 113 (representative feature calculation unit) calculates, for each similar posture image group generated by the similar posture image group generation unit 112, a representative appearance feature amount that represents the appearance feature amount of the person image constituting the group. In addition, a person image is cut out based on the mask image, the frame number, and the time series image, the person ID for each time series person image, the frame number list of each similar posture image group, the person image and the shape feature amount for each frame number, and each Person information 120 in which the representative appearance feature amount of the similar posture image group is associated is stored in the storage unit 12.

  The person information generation unit 113 calculates the representative appearance feature value by averaging the appearance feature values of the person images in each similar posture image group. The average number of person images may be used, but a specified number of person images close to the average value may be used. Alternatively, the appearance feature value closest to the average appearance feature value of the person images of each similar posture image group may be used as the representative appearance feature amount, or the clustering result in the similar posture image group generation unit 112 may be used as the representative feature value. The appearance feature amount corresponding to the cluster center of each similar posture image group can be referred to as the representative appearance feature amount. Further, the person information generation unit 113 may generate the representative shape feature amount of each similar posture image group in the same manner as the appearance feature amount with respect to the shape feature amount.

  The storage unit 12 includes, for example, a storage device such as a hard disk drive (HDD), a RAM (Random Access Memory), etc., and stores the personal information 120 generated by the personal information generation unit 113 as well as temporary calculation data. It is used by the signal processing unit 11 as a storage area (not shown). In addition, the storage unit 12 stores various programs and various data used in the signal processing unit 11, and inputs / outputs such information to / from the signal processing unit 11.

[Configuration of Person Identification Device 2]
FIG. 2 is a block diagram showing a schematic configuration of the person identification device 2 (collation unit). The person identification device 2 includes an input unit 20, a signal processing unit 21, a storage unit 22, and an output unit 23. As already described, the person identification device 2 performs an identification process for finding the same person using the person information 120 created by the person information generation device 1. In the present embodiment, a user requests a search for a person appearing in a time-series image of a certain monitoring camera (hereinafter referred to as camera A), and the person identification device 2 receives another monitoring camera (hereinafter referred to as camera B). It is assumed that the person is searched from the time-series images photographed in (1).

  The input unit 20 is an input device such as a keyboard and a mouse, and is connected to the signal processing unit 21 and is used when a user selects a person to be searched or designates a database (DB) to be searched.

  Similar to the signal processing unit 11, the signal processing unit 21 is configured using an arithmetic device such as a CPU, and is connected to the input unit 20, the storage unit 22, and the output unit 23. The signal processing unit 21 reads out and executes the program from the storage unit 22 and serves as a person information acquisition unit 210, a similar posture image group selection unit 211, a color conversion unit 212, a person similarity calculation unit 213, a collation result generation unit 214, and the like. Function.

  The person information acquisition unit 210 reads the person information (person information 220 </ b> A) of the person requested to be searched from the input unit 20 (hereinafter referred to as a query person) from the storage unit 22. In addition, the person information acquisition unit 210 receives from the storage unit 22 person information groups (person information 220B) of a plurality of persons (hereinafter referred to as DB (database) persons) designated by the monitoring unit as a collation partner from the input unit 20. Read. Then, the person information acquisition unit 210 outputs the person information 220 (220A, 220B) of the person specified as the search target or collation partner to the similar posture image group selection unit 211.

  The similar posture image group selection unit 211 has a similar posture image group having the most similar posture from the similar posture image group stored in the person information of the query person and the similar posture image group stored in the person information of each DB person. Select a pair. The similar posture image group selection unit 211 outputs personal information to which the information for specifying each selected similar posture image group is added to the color conversion unit 212.

  The similar posture image group pair is selected by comparing the similarity between the shape feature amounts of the similar posture image groups. Similarity between similar posture image groups is determined by using a matching method such as dynamic programming in the same manner as the method used by the similar posture image group generation unit 112 of the person information generation device 1. Can be obtained by comparing.

  The query person and the DB person are collated by collating similar posture image group pairs. By selecting and comparing human images with similar postures in this way, it is possible to suppress a decrease in collation accuracy due to differences in human postures and orientations.

  The color conversion unit 212 uses the color between the similar posture image group pairs selected by the similar posture image group selection unit 211 to absorb the difference in the lighting conditions between the person image of the query person and the person image of the DB person. The person information obtained by adding the person image subjected to the conversion process and color-converted is output to the inter-person similarity calculation unit 213. The color conversion process is performed by BTF (Brightness Transfer Function).

Specifically, the color conversion unit 212 calculates a cumulative color histogram of each person image of the query person and DB person, converts a luminance value (color luminance value) of one color component of the cumulative color histogram, and approximates the other. Each color luminance value is converted so that Each cumulative color histogram is calculated using a plurality of frames of human images constituting a similar posture image group. That is, one cumulative color histogram is a function H ₁ (B ₁ ) for the color luminance value B ₁ , the other cumulative color histogram is a function H ₂ (B ₂ ) for the color luminance value B ₂ , and a color luminance value B ₁ by BTF. Is expressed as f (B ₁ ), the color conversion unit 212 defines the function f so that H ₁ (B ₁ ) matches H ₂ (f (B ₁ )). Cumulative frequency _{H 1} of the color conversion section 212 and the cumulative frequency _{H 2 (B 2)} to the luminance values 0 to B ₂ for each luminance value _{B 2,} until the luminance values 0 to B ₁ for each luminance value _{B 1} ( B ₁ ) is obtained, and a conversion f (B ₁ ) is performed in which each of the luminance values B ₁ is replaced with a luminance value B ₂ that satisfies H ₁ (B ₁ ) = H ₂ (B ₂ ). This conversion is performed for each color component. In practice, since the luminance value and frequency are discrete, in general, conversion that completely satisfies H ₁ (B ₁ ) = H ₂ (B ₂ ) cannot be obtained, and the color conversion uses one accumulated color histogram for the other. The conversion approximates the color histogram. Here, the change characteristic represented by the curve (broken line) of the cumulative histogram is called an integral characteristic, and approximating the characteristic by BTF is expressed as integrating the integral characteristic.

  Since the color conversion performed by the color conversion unit 212 is performed on a human image excluding the background, the difference in the background color distribution does not affect the color conversion. In addition, the color conversion performed by the color conversion unit 212 is conversion that matches the integration characteristics of the person image of a specific person, and is conversion that imposes a constraint condition that does not change the order of the luminance values as described above. For this reason, even if other people with different original color distributions try to match the integration characteristics, they cannot be matched due to the constraint conditions, and the color distributions of both are not similar. However, if the principals having the same original color distribution try to match the integral characteristics, they can be matched even if there are constraint conditions, and the illumination conditions are corrected and the color distributions of both are similar.

  As a result, it is possible to widen the difference between the collation between the principals and the collation between others, and the collation accuracy can be improved.

  For example, if the query person and the DB person are the same, but there is a difference between the accumulated color histograms due to the difference in the lighting conditions, if color conversion is performed so as to match the integral characteristics, The color histograms are similar color histograms having similar peak positions and relative sizes. In other words, only the illumination condition is successfully corrected by the color conversion, and the skin color of both person images and the color of clothes are similar.

  On the other hand, for example, when the query person and the DB person are different persons whose clothes colors and skin exposures are different, if color conversion is performed so as to match the integral characteristics, the relative peaks of the respective colors in the color histograms of the two are compared. The difference in the relative positional relationship and relative intensity remains, and the color-converted human image has a distorted color distribution, but no correction for the difference in illumination conditions is expected.

  As described above, the color conversion described above opens a difference in the degree of color matching between the person and the other person, so that the person matching accuracy can be improved while correcting the illumination condition. In addition, since this color conversion is performed in units of similar posture image groups in which the posture / direction and lighting conditions of the person are aligned, the effect of widening the difference can be further enhanced.

  The inter-person similarity calculation unit 213 calculates the similarity (inter-person similarity) between the person image of the query person that has been subjected to color conversion by the color conversion unit 212 and the person image of each DB person. The person-to-person similarity is output to the matching result generation unit 214.

  The similarity between persons is calculated as the similarity of appearance feature values. Specifically, the inter-person similarity calculation unit 213 re-extracts the representative appearance feature amount from the color-converted person image constituting the similar posture image group of the query person that is one of the similar posture image group pairs, and The representative appearance feature value of the object image constituting the similar posture image group of each DB person who is the other of the similar posture image group pairs is read from the person information, and the representative appearance feature value of the query person and the representative appearance feature value of each DB person are read. Is calculated as the similarity between persons. The similarity of the RGB histogram, which is an appearance feature value, can be obtained by calculating a histogram intersection. Even when an HSV histogram or a luminance gradient histogram is used instead of the RGB histogram, these histogram intersections may be calculated. Further, when a plurality of types of feature quantities are used as appearance feature quantities, the similarity between the feature quantities can be weighted and added with a predetermined weight to obtain the similarity between persons.

  The collation result generation unit 214 determines a person whose DB similarity calculated by the person similarity calculation unit 213 is equal to or greater than a predetermined identification threshold as the same person as the query person. The person information 220 of the person is output to the output unit 23. On the other hand, if there is no DB person having a similarity between persons equal to or higher than the identification threshold, it is determined that there is no corresponding person and a message to that effect is output to the output unit 23.

  Similar to the storage unit 12, the storage unit 22 includes a storage device such as an HDD, and stores personal information 220 </ b> A and 220 </ b> B and is used by the signal processing unit 21 as a temporary storage area (not shown) for calculation. In addition, the storage unit 22 stores various programs and various data used in the signal processing unit 21, and inputs / outputs such information to / from the signal processing unit 21.

  Here, the person information 220A stores person information of a plurality of persons generated from the time-series images of the camera A by the person information generation apparatus 1, and the person information 220B stores the person information generation apparatus 1 in time series of the camera B. The personal information of a plurality of persons generated from the image is stored. In this embodiment, the case where there are two cameras has been described. However, when there are more than two cameras, the same number of person information 220 as the number of cameras is stored in the storage unit 22.

  The output unit 23 includes a display. The output unit 23 is connected to the signal processing unit 21 and outputs the result generated by the matching result generation unit 214.

  When the person information generation device 1 and the person identification device 2 are configured as an integrated person identification / search system, the signal processing unit 11 and the signal processing unit 21 can be configured by a common arithmetic device, and the storage unit 12 and the storage unit 22 can also be constituted by a common storage device.

[Operation of Person Information Generating Device 1]
FIG. 3 is a schematic flowchart for explaining the operation of the person information generating apparatus 1.

  The person tracking unit 110 performs background difference processing on each image of the time-series image input from the input unit 10 to extract a person image by removing background pixels from the image, and generates a mask image ( Step S10). Then, the person tracking unit 110 performs template matching between the person image of the current frame extracted in step S10 and the person images extracted up to the previous frame to associate person images of the same person with each other (step S11). The person tracking unit 110 assigns a common person ID to the person image of the same person, and assigns a new person ID to the person image of the new person. The person tracking unit 110 adds the frame number of the current frame, the mask image of the person image extracted in the current frame, and the person ID to the storage unit 12 in association with each other.

  The person feature extraction unit 111 extracts a shape feature amount from the person image extracted by the person tracking unit 110 in step S10, and adds it to the storage unit 12 in association with the person image (step S12). Also, the person feature extraction unit 111 extracts appearance feature values from the person image, and adds the appearance feature amount to the storage unit 12 in association with the person image (step S13).

  If there is a person who has finished tracking (YES in step S14), the process proceeds to step S15. If not (NO in step S14), the process in steps S10 to S13 is performed on a newly input frame. repeat. The person feature extracting unit 111 collects the image information of the tracking-completed person recorded in steps S <b> 10 to S <b> 13 and outputs it to the similar posture image group generation unit 112 and the person information generation unit 113.

  The similar posture image group generation unit 112 performs clustering on the time-series human images of the tracking-completed persons based on the shape feature amount extracted in step S12 while satisfying the temporal continuity requirement in the group. An image group is generated, and a frame number list for each group is output to the person information generation unit 113 (step S15). That is, the similar posture image group generation unit 112 divides the time-series human image into groups including continuous frames in which the image feature of the human image has a predetermined similarity.

FIG. 4 is an explanatory diagram illustrating an example of processing of the similar posture image group generation unit 112. The series person image 131 when similar posture image group generating section 112 for person tracking unit 110 generates divided by continuous duty time _{T C} seconds interval, divided time-series images 130 (130a, 130b, 130c) defines a. The similar orientation image group generation unit 112 divides the human image 132 in each divided time-series image 130 into groups of frames (frame groups) having similar shape feature amounts and temporally continuous. For example, the person images 132-1 to 132-3 in the divided time-series image 130a have similar shape feature amounts, and form one frame group 133-1. Of the human images 132-4 to 132-7 in the divided time series image 130b, the human images 132-4, 132-5, and 132-7 have similar shape feature amounts. Among them, the person images 132-4 and 132-5 have consecutive frame numbers and are set as a frame group 133-2. The person image 132-7 has a frame number different from the person images 132-4 and 132-5. Since it is not continuous, it is set as another frame group 133-3. Since the person image 133-6 is not similar in shape feature quantity to the person images on both sides with adjacent frame numbers, the frame group 133-4 is formed alone. Figures 132-7～132-9 is shape feature is similar and frame numbers are consecutive, the continuity constraint time _{T C} between the figures 132-7 and figures 132-8,132-9 Since it is divided, it is not made into one frame group, but is divided into separate frame groups 133-3 and 133-5.

  The person information generation unit 113 generates a representative appearance feature amount by averaging the appearance feature amounts of the person images constituting the group for each similar posture image group of the tracking-finished person generated in step S15 (step S15). S16). In step S15, the person information generation unit 113 uses the person ID of the tracking end person obtained in steps S10 to S12, the time-series person image of the person and each frame number, and the shape feature amount of each person image in step S15. The person information 120 of the tracking end person is generated by adding the information of the generated similar posture image group and the representative appearance feature value generated in step S16, and is stored in the storage unit 12 (step S17). When the person information 120 of the tracking end person is generated, the signal processing unit 11 returns the process to step S10 and proceeds to the process of the next frame.

[Operation of Person Identification Device 2]
FIG. 5 is a schematic flowchart for explaining the operation of the person identification device 2.

  The user operates the input unit 20 to select one of the persons shown in the time-series image of the camera A as a query person (step S20), and selects the camera B as a camera to be associated ( Step S21). The selection of the camera B is equivalent to selecting all persons shown in the time-series image of the camera as DB persons.

  The person information acquisition unit 210 reads the person information 220A of the query person specified in step S20 from the storage unit 22 (step S22).

  In the subsequent loop processing of steps S23 to S28, the person similarity between the query person and all DB persons is calculated.

  The person information acquisition unit 210 sequentially reads the person information 220B of each DB person from the storage unit 22 (step S23). Then, the similar posture image group selection unit 211 compares the shape feature amount of each similar posture image group of the query person with the shape feature amount of each similar posture image group of the DB person read in step S23, and the shape feature amount. A pair of similar posture image groups with the most similar is selected (step S24).

  The color conversion unit 212 calculates a cumulative color histogram of each of the similar posture image group of the query person and the similar posture image group of the DB person constituting the pair selected in step S24 (step S25), and the similar posture image of the query person. A color conversion process for approximating the accumulated color histogram of the group to the accumulated color histogram of the similar posture image group of the DB person is performed on each person image constituting the similar posture image group of the query person (step S26).

  The person-to-person similarity calculation unit 213 extracts a representative appearance feature value from the similar posture image group of the query person that has undergone color conversion in step S26, and the representative appearance feature value and the similar posture image group selected for the DB person. The similarity with the representative appearance feature value is calculated as the similarity between the query person and the DB person (step S27).

  The person similarity calculation unit 213 checks whether or not the person similarity is calculated for all DB persons specified in step S21 (step S28). If not calculated for all (NO determination in step S28), the processes in steps S23 to S27 are repeated for the uncalculated DB person. On the other hand, if the inter-person similarity is calculated for all DB persons (YES in step S28), the process proceeds to step S29.

  The matching result generation unit 214 selects the maximum person similarity from among the person similarities calculated for each DB person by the loop processing in steps S23 to S28, compares the person similarity with the identification threshold, and determines the maximum person similarity. Is equal to or greater than the identification threshold (YES in step S29), the DB person whose inter-person similarity is calculated is determined to be the same person as the query person, and the person information of these persons is output to the output unit 23. Output (step S30).

  If the maximum similarity between persons is less than the identification threshold (NO determination in step S29), the matching result generation unit 214 outputs to the output unit 23 that the DB person corresponding to the query person has not been detected. S30 is omitted.

  The output unit 23 outputs the person information of the same person or the information of no corresponding person input from the collation result generation unit 214 to the display or the like as the identification process result (step S31), and ends the process.

[Modification]
(1) In the above embodiment, when after the image sequence is divided into the divided time-series images in continuity duty time T _C seconds intervals to define a similar orientation group in the divided time-series image. On the other hand, it is also possible to first find a group of human images having similar postures and continuous frames in a time series image and define it as a similar posture group. In this case, if the number of consecutive frames with similar postures increases, the width of the illumination variation within the frame group also increases, and the image characteristics of the human image can be dull. Correspondingly, as in the above embodiment, by introducing a continuous duty time T _C, for successive groups time similar orientation exceeds T _C, the group after the division by dividing each example T _C, respectively Similar posture groups can be used.

  (2) In the above embodiment, the similar posture image group generation unit 112 generates continuous frames having similar shape feature amounts. In another embodiment, the similar posture image group generation unit 112 generates a continuous frame in which the shape feature value and the appearance feature value are similar. In this case, for example, the similar posture image group generation unit 112 obtains the similarity of the appearance feature amount in addition to the similarity of the shape feature amount between temporally adjacent human images in the divided time series image, and obtains these in advance. A similar posture image group is generated by clustering based on the similarity obtained by weighted addition with the set weight. Further, for example, the similar posture image group generation unit 112 clusters the divided time series images based on the shape feature amount, and further clusters the clustering results based on the appearance feature amount to generate a similar posture image group.

  DESCRIPTION OF SYMBOLS 1 Person information generation apparatus, 2 Person identification apparatus, 10,20 Input part, 11,21 Signal processing part, 12,22 Storage part, 23 Output part, 110 Person tracking part, 111 Person feature extraction part, 112 Similar posture image group Generation unit, 113 person information generation unit, 210 person information acquisition unit, 211 similar posture image group selection unit, 212 color conversion unit, 213 interperson similarity calculation unit, 214 collation result generation unit

Claims (3)

A moving object collating device for collating whether or not a plurality of time-series object images each having a moving object imaged are of the same moving object,
A continuous frame generation unit that divides the time-series object image into continuous frames in which image characteristics of the moving object are similar to each other;
A representative feature calculator that calculates a representative feature representative of a luminance feature of the moving object in a plurality of frames constituting the continuous frame;
A collating unit that collates the plurality of time-series object images with respect to the representative feature for each continuous frame, and determines that the time-series object images including the continuous frames that have been collated and matched are those of the same moving object;
A moving object matching device comprising: The moving object matching device according to claim 1,
The said representative feature calculation part calculates the said representative feature by averaging the luminance feature of the said moving object in the some flame | frame which comprises the said continuous frame, The moving object collation apparatus characterized by the above-mentioned. In the moving object collation device according to claim 1 or 2,
The said collating part selects the said continuous frame with which the shape characteristic of the said moving object is the most similar between the said time-sequential object images, and collates the selected said continuous frame, The moving object collation apparatus characterized by the above-mentioned.

Images