JP2010239992A - Person identification device, person identification method, and person identification program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To highly precisely identify a person according to the situation of the person included in an image. <P>SOLUTION: A person identification device 10 identifies the person included in videos photographed in different periods, and includes: an identification element extraction means 15 which extracts the identification element of the person; an identification element selection means 16 which selects an identification element not to be used from among already identification elements according to a photographic condition preset based on a feature quantity obtained by the identification element extraction means; and an identification means 17 which identifies the person using the identification element obtained by the identification element selection means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

The present invention relates to a person identification apparatus, a person identification method, and a person identification program, and in particular, a person identification apparatus, a person identification method, and a person for identifying a person with high accuracy according to the situation of a person included in an image It relates to an identification program.

Conventionally, a person is detected by using an image or video taken by an imaging means such as a camera, and the person is identified by face authentication, or obtained in time series when tracking a person. There is known a method for determining whether a person has moved from an image group or another person and identifying the person (see, for example, Patent Documents 1 and 2).

In the above-mentioned Patent Document 1, a face verification is performed by detecting a three-dimensional shape of a person's face and an image of the surface thereof, comparing the three-dimensional shape of the face and the image of the surface with inquiry face image data, and identifying the person. In the method, the photographing condition of the inquiry face image data is estimated, the face image data for matching is generated by using the graphics means based on the photographing condition, the three-dimensional shape and the image of the surface, and the face image data for matching and the inquiry face image A method is shown in which the data is compared, and when the inquiry face image data has a small difference from the face image data for collation, the face image data for collation is identified as a person.

Moreover, in the above-mentioned patent document 2, it is a person identification method using the image matched with the audio | voice, Comprising: The face reflected in the 1st image and the face reflected in the 2nd image are the same A first determination step for determining whether or not a person is present, a second determination step for determining whether or not sounds attached to the first and second images belong to the same person, and a first determination step Or the person identification method which has the 3rd determination process which determines that the image determined as the same person by either of the 2nd determination processes is the same person is shown.

JP 2001-283224 A Japanese Patent Laid-Open No. 2000-76459

However, in the case of the above-described method, a person is identified by using face authentication and a person is identified, but a face is not always shown in a captured image or video. In other words, the person on the image must be tracked as a predictive crime crime by the image, and whether or not it is suspicious changes depending on the target person, so the action etc. are always tracked regardless of the orientation of the face. There is a need for an identification method according to the shooting situation included in the shot image.

The present invention has been made in view of the above-described problems, and a person identification device, a person identification method, and a person identification program for accurately identifying a person according to the situation of a person included in an image The purpose is to provide.

In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

According to the present invention, in a person identification device for identifying a person included in videos taken at different times, an identification element extraction unit that extracts an identification element of the person, and a feature amount obtained by the identification element extraction unit in advance. Identification element selecting means for selecting an unused identification element from identification elements already set according to set photographing conditions, and identification for identifying the person using the identification element obtained by the identification element selection means Means. Thereby, a person can be identified with high accuracy according to the situation of the person included in the image.

In the present invention, the identification means resets the weights for the identification elements selected by the identification element selection means in correspondence with the current weighting ratio.

In the present invention, the identification element extraction means extracts gaits, body types, and genders as feature quantities as the identification elements, and the gait includes a step length, a pace, a walking speed, a standard deviation of a walking speed, and a walking acceleration. And at least one of a standard deviation, a body inclination, and a leg length.

Further, in the present invention, when the identification unit identifies that the person is the same person, a tracking unit that tracks the person, and a suspicious person detection unit that detects a suspicious person from the tracking result in the tracking unit, And screen generating means for generating a screen for displaying the tracking result.

Furthermore, in the present invention, in a person identification method for identifying a person included in images shot at different times, an identification element extraction procedure for extracting the identification element of the person, and a feature amount obtained by the identification element extraction procedure An identification element selection procedure for selecting an unused identification element from identification elements that are already set according to preset imaging conditions, and the person is identified using the identification element obtained by the identification element selection procedure. And an identification procedure. Thereby, a person can be identified with high accuracy according to the situation of the person included in the image.

In the present invention, the identification procedure resets the weights for the identification elements selected by the identification element selection procedure in correspondence with the current weighting ratio.

In the present invention, the identification element extraction procedure extracts, as the identification element, a gait, a body shape, and gender as feature quantities, and the gait includes a step length, a pace, a walking speed, a standard deviation of a walking speed, and a walking acceleration. And at least one of a standard deviation, a body inclination, and a leg length.

Further, in the present invention, when the identification procedure identifies the same person as the person, a tracking procedure for tracking the person, and a suspicious person detection procedure for detecting a suspicious person from the tracking result in the tracking procedure, And a screen generation procedure for generating a screen for displaying the tracking result.

Further, according to the present invention, in a person identification program in a person identification device for identifying a person included in images taken at different times, a computer is provided with an identification element extraction means for extracting the identification element of the person, and the identification element extraction means. Using the identification element selection means for selecting an identification element not to be used from the identification elements already set according to the imaging conditions set in advance from the obtained feature quantity, and using the identification element obtained by the identification element selection means And function as identification means for identifying the person. Thereby, a person can be identified with high accuracy according to the situation of the person included in the image. Further, by installing the program, the person identification process in the present invention can be easily realized by a general-purpose personal computer or the like.

In the present invention, the identification means resets the weights for the identification elements selected by the identification element selection means in correspondence with the current weighting ratio.

In the present invention, the identification element extraction means extracts gaits, body types, and genders as feature quantities as the identification elements, and the gait includes a step length, a pace, a walking speed, a standard deviation of a walking speed, and a walking acceleration. And at least one of a standard deviation, a body inclination, and a leg length.

Further, in the present invention, when the identification unit identifies that the person is the same person, a tracking unit that tracks the person, and a suspicious person detection unit that detects a suspicious person from the tracking result in the tracking unit, And screen generating means for generating a screen for displaying the tracking result.

According to the present invention, it is possible to identify a person with high accuracy according to the situation of the person included in the image.

It is a figure which shows an example of schematic structure of the person identification apparatus in this embodiment. It is a figure which shows an example of how to walk a person. It is a figure which shows the value used as the reference | standard at the time of calculating a gait feature-value. It is a figure for demonstrating the example of calculation of an obesity degree. It is a figure for demonstrating the content of the extraction element corresponding to the feature-value used for person identification in this embodiment. It is a figure which shows an example of the state determination in this embodiment. It is a figure which shows the example which adjusted the weight of the feature-value. It is a figure for demonstrating the background difference method using the parameter strong against a sunlight change. It is a figure for demonstrating the statistical background difference method which considered the presence of the person. It is a figure which shows an example of a tracking screen. It is a figure which shows an example of the hardware constitutions which can implement | achieve the person identification process in this embodiment. It is a flowchart which shows an example of the person identification processing procedure in this embodiment. It is an example flowchart which shows a person identification process procedure.

<About the present invention> In the present invention, a person is identified by combining a plurality of ambiguous identification elements (height, figure, walking, gender, gait, etc.) as in the process when a person recognizes a person. Build a method. Specifically, in the present invention, an appropriate identification element is selected according to the captured image or the shooting situation of the person, and the identification process is performed based on the selected condition.

That is, the present invention selects an optimal feature without using an inappropriate feature when identifying a person from the feature. Specifically, a person's state (stopped, walking, whether there was sunshine change, etc.) is determined, and based on the results, various feature quantities (step length, walking speed, clothing color information, face authentication, etc.) It is determined whether or not a feature amount related to a person that can be extracted by image processing such as a result is used for person identification. The degree of separation is calculated only for the feature amount determined to be used, weighting is performed based on the degree of separation, and the contribution rate for each feature amount to person identification is changed. Thereby, selection of the feature amount according to the situation is performed.

Below, the form which implemented suitably the person identification device in this invention, the person identification method, and the person identification program is demonstrated using drawing.

<Schematic Configuration Example of Person Identification Device> FIG. 1 is a diagram illustrating an example of a schematic configuration of a person identification device according to the present embodiment. 1 includes an input means 11, an output means 12, a storage means 13, a person detection means 14, an identification element extraction means 15, an identification element selection means 16, an identification means 17, The tracking unit 18, the suspicious person detection unit 19, the screen generation unit 20, the notification unit 21, the transmission / reception unit 22, the control unit 23, and the imaging unit 24 are configured. Note that the transmission / reception unit 22 is connected to an imaging unit 24 such as a camera that captures a predetermined region, and can acquire each time-series image included in the video captured by the imaging unit 24. Further, the imaging unit 24 may be configured integrally with the person identification device 10.

The input means 11 receives various instructions such as a person detection instruction from the user, an identification element extraction instruction, an identification element selection instruction, an identification instruction, a tracking instruction, a suspicious person detection instruction, a screen generation instruction, a notification instruction, and a transmission / reception instruction. Accept. Note that the input unit 11 includes, for example, a keyboard, a pointing device such as a mouse, a voice input device such as a microphone, and the like.

The output means 12 includes the instruction content input by the input means 11, control data generated based on each instruction content, person detection means 14, identification element extraction means 15, identification element selection means 16, identification means 17, tracking The contents of various information obtained by the progress or result executed by the components such as the means 18, the suspicious person detection means 19, the screen generation means 20, the notification means 21, and the transmission / reception means 22 are displayed and the sound is output. The output unit 12 has a screen display function such as a display, a sound output function such as a speaker, and the like.

The storage unit 13 can store various information for realizing the above-described embodiment, and reading and writing are performed as necessary. Specifically, the storage unit 13 is configured to track the person detection result in the person detection unit 14, the identification element extraction result in the identification element extraction unit 15, the identification element selection result in the identification element selection unit 16, the identification result in the identification unit 17, and the tracking. The tracking result in the means 18, the suspicious person detection result in the suspicious person detection means 19, the screen information generated in the screen generation means 20, the contents notified by the notification means 21, the contents notified so far, the transmission / reception means 22 Information, information controlled by the control means 23, error information when an error occurs, log information, and a program for realizing the present invention are stored.

The person detection unit 14 acquires a video shot by the imaging unit 24 such as a camera, and among the time-series images included in the acquired video, a predetermined image (interval for each frame image or several frames). The face of one or a plurality of persons is detected. The person detection unit 14 compares, for example, consecutive image frames, and there is a place where the color (luminance, chromaticity, etc.) changes within a predetermined time, and the person detection means 14 further has a place where the place is greater than the predetermined area. Detect as a person.

Other human detection methods include, for example, a method of extracting an area that matches the shape of a person using edges or the like, a method of using a thermal image to set a heat source different from the background to a human body region, and a plurality of cameras A method for detecting human-like objects by recognizing shape and size using a human, a method for detecting a human body by learning in advance using silhouette images etc. for human movement (like walking), etc. Can be used. Note that the person detection method in the person detection means 14 is not limited to this in the present invention.

The identification element extraction unit 15 extracts an identification element from a video (image) obtained by the imaging unit 24 or the like. The identification elements in the present embodiment include, for example, gaits, body shapes, sexes, and the like, and feature values are extracted using each as a parameter for person identification. A specific extraction method will be described later.

The identification element selection unit 16 selects a type used as a determination criterion for determining whether or not the persons detected in the preceding and following frames in the identification unit 17 are the same. Specifically, the identification element selection means 16 captures the time and place taken by the imaging means 24, the appearance of the person (whether the face is reflected, or the whole body is photographed with respect to at least one identification element set in advance. In other words, a part of the identification element set in advance according to whether or not it is used is not used. The identification element selection unit 16 may newly extract an identification element according to the above-described conditions.

That is, the identification element selection unit 16 determines whether or not each feature amount is effective, and when it is determined that a certain feature amount is not effective according to the state of the person, in the subsequent identification processing The feature amount can be avoided. For example, the identification element selection means 16 does not use such color information because the color of clothes and the color of hair change when the lighting conditions change, as in the case of performing person identification between day and night. Like that. A specific description of the identification element selection unit 16 will be described later.

Based on the identification element extracted by the identification element extraction means 15 and the content selected by the identification element selection means 16, the identification means 17 is detected from two images extracted from videos taken at different times. The identification process of whether or not the same person is included is performed on one or more persons.

Specifically, the identification unit 17 identifies a person using an appropriate parameter by a feature amount selection method according to the state of the image. For example, the identification unit 17 determines a person's state (stopped, walking, whether there was a change in sunshine, etc.), and based on the result, various feature amounts (step length, walking speed, clothes color information, etc.) Is used for person identification.

The identification unit 17 calculates the degree of separation only for the feature amount determined to be used, performs weighting based on the degree of separation, and changes the contribution rate for each feature amount to person identification. That is, the identification unit 17 resets the weights for the identification elements selected by the identification element selection unit 16 in correspondence with the current weighting ratio. Thereby, the person determination using the feature amount according to the situation can be performed.

The tracking unit 18 captures the video obtained from the imaging unit 24 and tracks the behavior of the same person from the time-series images. The tracking means 18 tracks a person from the foot and head positions of the person area included in the image and their characteristics, and when the size of the person that can be determined to be the same changes, a part of the person may be a building or the like. It is determined whether or not it is concealed.

The part of the person to be concealed is, for example, the lower body, the upper body, the head, arms, and the torso. That is, in this embodiment, when information about the height of the human body from the toes of the feet to the head can be acquired at least once on the screen, if both the head and feet are not simultaneously concealed, then the head or If the movement range in the next time-series image of the foot is within a predetermined range and the color information is the same, it can be assumed that there is a person in the concealed part by assuming that the person is the same person and tracking that person it can. The tracking unit 18 tracks one or more persons displayed in the image over time.

Furthermore, the tracking means 18 is also provided with a function for estimating a person area. Specifically, the tracking means 18 is a person for estimating on the screen that the person area is estimated from the outside area of the human body existing in the image, and that the person exists in the area frame. Estimate the region.

Further, the tracking means 18 estimates the correct human body region when it is determined that there is a portion concealed by a part of the person due to the shielding object, for example. In this case, the tracking means 18 displays the outline (silhouette) or region of the human body part that is concealed on the screen. Thereby, the position where the person who is not displayed on the image by a concealment object exists can be grasped | ascertained correctly.

Further, when tracking the person present in the image over time, the tracking unit 18 can estimate the next operable range from the orientation, posture, and behavior so far. In this case, the tracking unit 18 generates information for framing the outer frame of the maximum range in which the person can operate next and compositing it into an image.

The suspicious person detection means 19 compares the time-series behavior data or the like stored in the storage means 13 or the like in advance with the tracking result obtained by the tracking means 18 or the like. Detection is performed when is a suspicious person. In addition, the tracking means 18 may detect the person being suspicious if there is a suspicious behavior such as the person being tracked hidden behind a shield, being worried about the camera for more than a predetermined time, or being scrambled. Detect as. The tracking unit 18 notifies the notification unit of the detected suspicious person information.

For example, the suspicious person detecting means 19 has at least one suspicious behavior such as the person to be tracked hidden behind a shield, taking care of the camera for more than a predetermined time, scrambled, or staying for a long time. In that case, the person is detected as a suspicious person. Furthermore, the suspicious person detection means 19 detects as a suspicious person even when the person to be tracked is wearing a mask or wearing sunglasses to hide his face. The suspicious person detecting means 19 notifies the notifying means 21 of the detected suspicious person information.

The screen generation unit 20 generates a screen that displays the video imaged by the imaging unit 24. Further, the screen generation unit 20 reflects the person area estimated by the tracking unit 18 in a time-series image included in the video currently acquired by the imaging unit 242 or the like. Specifically, the screen generation means 20 forms a frame outside the area estimated as the person area, combines it with the photographed video, and causes the output means 12 to display it. The screen generation means 20 displays an image obtained by screen generation on the output means 12 such as a display. At this time, the screen generation means 20 can display, for example, a numerical value of the position information of the frame of the person area on the screen.

In addition, the screen generation means 20 can estimate the moving range of the person over time required when tracking the person and display it on the screen. Furthermore, when the screen generation means 20 corresponds to the action pattern of a suspicious person set in advance, the screen generation means 20 generates a screen for notifying the content to that effect. Various kinds of information necessary for screen generation can be used by appropriately reading out necessary information from information stored in the storage unit 13 in advance. The generated screen is output together with sound and the like by the output means 12 having a display, a speaker, and the like.

The notification means 21 causes the screen generation means 20 to generate an image detected as a suspicious person obtained by the suspicious person detection means 19 and information related to the image (detection date and time, detection location, video for a predetermined time before). To display. In addition, the notification means 21 has an alert function for notifying a manager, a security officer, a supervisor, a representative manager, etc. of the building in response to a problem occurrence signal in such suspicious behavior detection. .

The transmission / reception means 22 receives monitoring video from one or a plurality of imaging means 24 via a communication network such as a LAN (Local Area Network) or the Internet. Note that the transmission / reception unit 22 does not have to receive the monitoring video directly from the imaging unit 24. For example, the video acquired in advance by the imaging unit 24 is temporarily stored somewhere, and the stored information is used. In this embodiment, person identification may be performed. Moreover, the transmission / reception means 22 can be used as a communication interface for transmitting to other terminals constituting the person identification apparatus 10 and receiving various data from other terminals.

The control means 23 controls the entire functional configuration of the person identification device 10. Specifically, the control unit 23 detects a person, extracts an identification element, selects an identification element, or performs an identification process based on input information from a user or the like input by the input unit 11. , Tracking processing, suspicious person detection processing, screen generation, communication control, and transmission / reception control. <Identification Element Accumulation> Here, the identification element extraction contents in the identification element extraction means 15 will be described. In the accumulation of identification elements, in this embodiment, three identification elements of “gait”, “body type”, and “gender” are extracted.

<Gait> First, a gait that is one of identification elements will be described. In general, since there are individual differences in the way people walk, the characteristics of how to walk are extracted from the person area detected from the captured video. FIG. 2 is a diagram illustrating an example of how a person walks. As shown in FIG. 2, when the person 30 moves, there is a unique way of walking. For this reason, the feature amount can be parameterized to be an element of identification detection.

<Regarding Gait Feature Amount> Here, for the feature amount in the present embodiment, for example, “step (Pixel / Step)”, “step (Min)”, “step speed (Pixel / Sec)”, slide Indicators for detecting foot walking, “standard deviation of walking speed (Pixel)”, “walking acceleration (Pixel / Sec ^ 2)”, indicators for detecting whether or not walking is jumping, “standard deviation of height ( At least one of “Pixel” ”,“ Body inclination (degree) ”representing the posture during walking,“ Leg length ”, and the like can be used.

Here, each feature amount described above can be calculated by the following mathematical formula. The “step length L (Pixel / Step)” is calculated by the following equation (1), the “step C (Step / Min)” is calculated by the following equation (2), and the “step speed S (Pixel / Sec) is calculated. ) ”Is calculated by the following equation (3),“ standard deviation of walking speed ss (Pixel) ”is calculated by the following equation (4), and“ walking acceleration A (Pixel / Sec ^ 2) ”is It is calculated by the following equation (5), “height standard deviation HS (Pixel)” is calculated by the following equation (6), and “body inclination (degree) θ” is shown below (7) “Leg length LL (Pixel)” is calculated by the following equation (8). In equations (1) to (8), M is the total number of frames in which the width w is maximized, and N is the total number of frames.

FIG. 3 is a diagram illustrating a reference value when calculating the gait feature amount.

In the relationship between FIG. 3 and the above-described equations (1) to (8), h represents the height of the person (height on the image), w represents the width (width on the image), and Ll represents the length of the left foot. Lr represents the length of the right foot, θ represents the inclination with respect to the axis perpendicular to the ground, H represents the center of gravity of the human head, G represents the center of gravity of the human area, and bx represents The distance from the center of the position of the left and right feet of the person 30 to the image frame 31 in the traveling direction of the person 30 is indicated, and by indicates the distance from the ground to the image frame 31 in the vertical direction.

Further, mj represents all frames that maximize the width w, ti represents the time when the frame i was photographed, and xi represents bx and by in the frame i.

The identification element extraction means 15 uses the value calculated from these mathematical expressions as a gait feature amount. Of the gait feature quantities described above, the identification ability is particularly high in the order of “height”, “leg length”, “walking speed”, and “step length”. Further, regarding the “body inclination”, although the degree of separation was low, when visually confirmed on a walking moving image, a significant difference is clearly shown for a person walking in a forward leaning posture. For this reason, it can be understood that this is an effective feature amount depending on the situation, such as applying after narrowing down to several people at the time of person identification.

<About Body Shape and Gender> Next, a body shape which is one of identification elements will be described. Here, the person is classified into three types of “skin”, “normal”, and “obesity”. It should be noted that since the determination criteria differ between men and women, it is preferable to also determine “sex”.

Here, it was found from prior experiments that attention was paid to, for example, the chest, abdomen, buttocks, body thickness, roundness of the body, and thinness of the legs when determining the body type. Thus, the feature amount “obesity level” is set by paying attention to the edge information of the chest and abdomen. Then, the “obesity degree”, “aspect ratio”, and “inertia main axis” representing the direction in which the object is extended are used as the feature quantities for body type determination. For gender determination, the aspect ratio, the principal axis of inertia, and the second moment were used in consideration of the influence of the chest that seems to be the most different between men and women.

Here, FIG. 4 is a diagram for explaining an example of calculating the degree of obesity. For example, as shown in FIGS. 4A to 4C, the degree of obesity is calculated by calculating the directional component of the edge at the same time as detecting the edge on the image, and then applying a Kirsch filter to determine the number of edges in each direction. Is calculated.

Therefore, specifically, the degree of obesity is calculated by the following equation (9).

In Equation (9) described above, Ex represents the number of edges in the x degree direction, and w represents the person width.

The aspect ratio, which is the average value of the person width for the upper body divided by the length of the person's upper body, can be calculated by the following equation (10), and the secondary moment / inertia principal axis is the image I (x , Y) where the (m + n) -order moment is μmn, the following equation (11) is obtained, and the secondary moment and the principal axis of inertia are calculated by the following equations (12) and (13). .

In the above-described equation (10), h indicates the height of the person. Gender is determined by extracting facial feature values from the face image in the image and comparing them with pre-set gender-specific feature parameters, or extracting feature values such as whole body lines from the image. Thus, the gender determination can be easily performed by comparing with the characteristic parameter for each gender set in advance.

<Identification Method in Identification Unit 17> Next, an identification method in the identification unit will be described with reference to the drawings. Conventional identification methods simply synthesize a plurality of feature quantities and determine them as one feature quantity. However, there is a high possibility of erroneous determination when noise or variation exists in some feature amounts. In addition, for example, when the height changes due to partial hiding of the body, the feature quantity may become meaningless in the first place.

In addition, since the classification ability of a person is different for each feature amount, there is a problem that the accuracy of determination is lowered by simply combining them. These effects are likely to increase as the feature amount handled increases. Therefore, in the present embodiment, in order to solve the above-described problem, “feature amount used for identification”, “feature amount selection condition”, and “person determination” are performed with the following contents.

<Regarding Feature Amount Used for Identification> FIG. 5 is a diagram for explaining the content of an extraction element corresponding to a feature amount used for person identification in the present embodiment. As shown in FIG. 5, the feature quantity names are “step length”, “walking speed”, “leg length”, “walking posture”, “hair color”, “clothing color”, “body type”, “sex”, “ At least one of “height” and the like is used, and person identification is performed using the extraction element shown in FIG. 5 corresponding to the feature amount. For example, as shown in FIG. 5, if the feature amount is “step length”, the extraction element is an average value of the peak values of the person rectangle width. In the case of “sex”, the extracted elements are the aspect ratio of the human body region, the inertial principal axis, and the second moment.

<Regarding the selection condition of feature amount> Next, when there is noise or variation in some feature amounts, which is the conventional problem described above, there is a high possibility of erroneous determination. When the height changes due to partial hiding, etc., the feature quantity is selected and selected in order to cope with the case where the feature quantity has no meaning in the first place. That is, in order to deal with the above-described problem, it is determined whether each feature amount is valid. In this state determination, when it is determined that a certain feature amount is not effective according to the state of the person, the feature amount is not used for the subsequent identification processing. For example, when the lighting conditions change, such as when it is from day to night, the color of clothes and the color of hair change, so that correct identification is not performed. Therefore, color information or the like is not used in the above case.

Here, FIG. 6 is a diagram illustrating an example of state determination in the present embodiment. As shown in FIG. 6, “state determination” and its “determination method”, and “invalidated feature amount” when the determination result is applicable, are described. The determination conditions shown in FIG. 6 are stored in advance in the storage means 13 and the like.

For example, in FIG. 6, the “state determination” is moving in the vertical direction and the oblique direction with respect to the camera to be photographed, and the “determination method” is a change in position information in the preset Y direction at that time. If there is, the stride length / leg length / walking posture / body type / sex among the feature amounts acquired by the identification element extraction means 15 is set as “invalidated feature amount”.

In addition, as the “state determination”, the determination that the user is stopped can be confirmed by a change in the position information of the person seen over time. As the “determination method”, when the position information has changed, The walking speed is defined as “invalid feature”.

Further, as the “state determination”, the determination that the change in sunshine is large can be confirmed by the change in the average luminance value of the monitoring area. To be featured ". An example of background difference using parameters that are resistant to changes in sunshine used in this embodiment will be described later.

Furthermore, as the “state determination”, the determination that the value greatly changes due to noise can be confirmed by the noise result based on the threshold when determining the threshold for each feature amount, and in that case, the corresponding feature amount Is set as “invalidated feature amount”.

In this way, by using this embodiment, when some feature quantities that have been impossible to identify until now or whose accuracy has been significantly reduced become abnormal values due to noise, or as feature quantities in the first place. Even when the meaning is lost, identification can be performed with high accuracy.

<Person Determination Method> Next, the person determination method in the present embodiment will be described with reference to the drawings. The classification ability of a person differs for each feature amount described above, and the accuracy of determination is reduced simply by combining determination conditions. Therefore, in order to cope with this problem, in the past, determination was performed using a simple distance between the average value of each feature value of each person recorded in the registered person information and each feature value of the input data. .

However, since the contribution rate to the identification differs for each feature amount, if it is simply combined, there is a possibility that the accuracy of the determination is lowered. Therefore, in the calculation of the distance, according to the contribution rate to the identification, the weight is given and calculated.

More specifically, the degree of separation is calculated for each feature amount using the registered feature amount of the person as an index of the person's classification ability, and the distance is calculated by assigning a weight according to the degree of separation. The person with the smallest calculated distance is set as the determination result. If it is determined that some of the feature values are not valid, the weight of the feature value is set to 0, and the weight is recalculated, thereby enabling highly accurate person determination.

Here, FIG. 7 is a diagram illustrating an example in which the weight of the feature amount is adjusted. As shown in FIG. 7, height, clothes color, hair color, and stride are set as registration information, and the respective weights are 50%, 5%, 15%, and 30%, with the total as 100%. For example, in the determination by Mr. A, when the determination is made between daytime and night, since the change corresponds to the sunshine change shown in FIG. Therefore, the respective weights for the selected identification element are reset according to the current weight ratio. Specifically, the height and stride weight, which were 50% and 30% so far, are set to 62.5% and 37.5%, respectively. Thereby, person determination can be performed without being affected by changes in sunshine or the like, and the accuracy of authentication can be improved. A specific example of the person extraction in the sunshine change used in the person determination of this embodiment will be described later.

Next, a distance calculation example including the state determination process in the present embodiment described above will be described. Here, regarding the degree of separation, as a general statistic, when the intra-class variance is small, the elements in each class are close to the average value of the class, and when the inter-class variance is large, each class Are present away from the overall average. Thereby, it can be said that each class is separated if the degree of separation is large.

Therefore, the overall average _{m i,} class n in average _{m n,} the entire dispersing sigma _t ^2, class n in dispersing sigma _n ^2, the number of classes n data when the omega _n, interclass variance sigma _b ² Is represented by the following equation (14), the intra-class variance σ _w ² is represented by the following equation (15), and the degree of separation is represented by the following equation (16).

The distance calculation method can be calculated by the following equation (17).

In the above-described equation (17), Da represents the weighted distance to the person a, St represents the degree of separation of the feature quantity t, and dt represents the simple distance to the person a with respect to the feature quantity t (however, the feature quantity) t is excluded from those not used in the state determination unit.).

<Person Extraction in Sunlight Change> Here, a background subtraction method using parameters that are resistant to sunshine change will be described as specific person extraction in the sunshine change described above. FIG. 8 is a diagram for explaining the background subtraction method using parameters that are resistant to changes in sunshine.

Since the background subtraction method, which is a conventional method, uses luminance information, it is effective in an environment where there is little change in illumination conditions, but there is a problem that erroneous detection frequently occurs in an environment involving changes in sunlight. In addition, since a threshold for determining whether the area is a background or a change area is required, there is a problem that a detection result depends on a threshold setter.

Therefore, in the present embodiment, first, parameters used for the background difference are selected, and RGB vectors (color vectors) are used. Here, the RGB vector is a parameter that is insensitive to changes in brightness in which color information (R, G, B) is treated as a vector, as shown in FIG. That is, since the angle between the background image and the vector of the current image calculated from the following equation (18) is used as the amount of change, only the region having a color configuration different from the background is not affected by the change in brightness. be able to.

In the above equation (18), v _x represents the RGB component at the pixel position (i, j) of the background image, and w _x represents the RGB component at the pixel position (i, j) of the current image.

In addition, by using the statistical background difference method for the background difference method, the presence or absence of a change pixel is determined probabilistically, and the setting of the threshold value is unnecessary. Thereby, the difference in the detection result by the threshold setting person was excluded. 8A shows the RGB vector of the person area, FIG. 8B shows the RGB vector of the background area, and FIGS. 8A and 8B show the RGB space in the pixel (i, j). Explains the angle formed by the above vector. FIG. 8B shows the vector change direction due to the brightness change for the current image.

FIGS. 8C and 8D show the difference in angle formed by the vectors in the RGB space in the pixel (i, j) due to the difference in brightness, and FIG. 8C shows the case of a bright image. FIG. 7D shows a dark video case.

Here, the background difference using the RGB vector described above is a case of a bright image as shown in FIG. 8C due to the characteristic that the vector angle is calculated from the color elements (R, G, B) constituting the pixel. The change amount Δθ (= α−β) in which the vector angle between the previous image and the current image has changed at the pixel position (i, j) is sufficiently small. However, as shown in FIG. 8D, a small vector (a pixel with low luminance) has a large vector angle that is sensitive to even a slight change such as noise, and the amount of change Δθ (= Since α−β) becomes a size that cannot be ignored, there is a possibility that false detections increase in dark images. Therefore, in this embodiment, the difference is performed by estimating the degree of influence due to noise according to the magnitude of the vector.

<Statistical Background Difference Method Considering Human Presence> Next, a statistical background difference method considering human presence will be described. FIG. 9 is a diagram for explaining the statistical background subtraction method considering the presence of a person. FIG. 9A shows a state of conversion from a person position in the image to a position in the real space, and FIG. 9B shows a statistical background subtraction method considering the presence of the person. .

In the conventional statistical background subtraction method, the appearance probability with respect to the luminance value I (0 to 255) serving as the background is calculated for each pixel in the image during initial learning. Then, for each pixel in the image input after learning, the appearance probability of the corresponding pixel obtained is referred to, whether it is a background or a change pixel is calculated by the above-described equation (18), and the result is determined. ing.

However, when a long time elapses after learning, the luminance value greatly changes, and there is a problem that it becomes impossible to separate the background and the change area. Accordingly, it is necessary to update the appearance probability of the luminance value of the background area (angle formed by the RGB vector) as needed. For example, the following equation (19) is used.

Note that the definition of each variable in the above equation (19) is that, at a certain pixel position (i, j), P (background | I) is the probability that the pixel position is the background when the pixel value is I. P (mobile object | I) indicates the probability that a moving object is present at the pixel position when the pixel value is I, and P (background) and P (mobile object) indicate that the pixel position is the background or moving object. The probability of being is shown. Moreover, since this probability is unknown, it is normally a fixed value of 0.5. As a result, the denominator and numerator cancel each other.

Further, P (I | background) indicates the probability that the pixel value is I when it is known to be the background, and an initial learning time is provided in the basic statistical background difference, and only the background is in between. This probability distribution is generated on the premise that an image is taken. P (I | mobile object) indicates the probability that the pixel value is I when it is known that the object is a moving object. This information is also usually not understood because the clothes and skin colors worn by people are different, and therefore gives an even probability distribution. That is, in the case of an image with 256 gradations, P (I | mobile object) = 1/256.

Here, in the above equation (19), when A ≧ 1, a background pixel is assumed, and when A <1, a change pixel is assumed. In the present embodiment, a statistical background difference in consideration of the person position detected in the previous frame in the above equation (19) is performed, and the person is tracked.

Specifically, as shown in FIG. 9A, the positional relationship with the camera in the real space is obtained based on the position of the person area obtained from the image. By applying this, the person area appearing in the current frame is predicted from the person area extracted in the previous frame.

That is, the person area obtained in the previous frame is converted to a position in the real space, and a range in which the person can move by the current time (a range in which a person can move in a predetermined time) is obtained. Furthermore, it is possible to estimate the person area appearing in the current frame by inversely converting the obtained movement range into the person area in the image. Therefore, the statistical information of the background area can be sequentially updated by setting the areas other than the estimated image area to be updated.

Further, as shown in FIG. 9B, by applying the above-described method (iii) of the present embodiment to the current image (i) rather than the general statistical background difference method (ii), a person can be obtained. It can be seen that the difference between the change pixel appearing as a region and the background pixel is clear.

<Tracking Screen in Tracking Unit 18> Next, the tracking screen in the tracking unit 18 will be described with reference to the drawings. FIG. 10 is a diagram illustrating an example of a tracking screen. As shown in FIG. 10, the tracking screen 40 is configured to include a video screen 41, an identification detection screen 42, and a tracking progress screen 43.

In addition, the video screen 41 and the identification detection screen 42 include a person area 44 in which an area corresponding to the person 30 is displayed in a rectangle having a vertical width and a horizontal width, person estimation information 45 estimated from the person area, and an actual captured image. Date information 46 is shown. Specifically, the person estimation information 45 displays whether or not tracking is performed, whether or not one is identified, height, width, area, and the like. The tracking progress screen 43 also includes an angle of view 47 of the imaging means 24 such as a camera that is capturing the video screen 41, a current position 48 of the person, a tracking route 49, and a predetermined time (for example, several seconds have elapsed). The movement range 50 of the person 30 in the latter part is displayed. Thereby, it is possible to accurately grasp from which position to which direction the camera is directed. Further, based on this content, the suspicious person detecting means 19 can detect whether or not the person being photographed is a suspicious person.

<Hardware Configuration Example> Here, the person identification device 10 described above can be controlled by a dedicated device configuration having the above-described functions, but an execution program (person identification) that can cause a computer to execute each function. Program), and the execution program is installed in, for example, a general-purpose personal computer, server, etc., the person identification process in the present invention can be realized.

A hardware configuration example of a computer capable of realizing the person identification process in the present embodiment will be described with reference to the drawings. FIG. 11 is a diagram illustrating an example of a hardware configuration capable of realizing the person identification process in the present embodiment.

11 includes an input device 51, an output device 52, a drive device 53, an auxiliary storage device 54, a memory device 55, a CPU (Central Processing Unit) 56 that performs various controls, and a network connection device. 57 are connected to each other by a system bus B.

The input device 51 has a keyboard and a pointing device such as a mouse operated by a user or the like, and inputs various operation signals such as execution of a program from the user or the like.

The output device 52 has a monitor for displaying various windows and data necessary for operating the computer main body for performing the processing according to the present invention, and displays the program execution progress and results by the control program of the CPU 56. can do.

The input device 51 and the output device 52 may be integrated input / output means such as a touch panel. In this case, the input device 51 and the output device 52 are predetermined using a finger of a user, a pen-type input device, or the like. The position can be touched to input.

Here, the execution program installed in the computer main body in the present invention is provided by a portable recording medium 58 such as a USB (Universal Serial Bus) memory or a CD-ROM, for example. The recording medium 58 on which the program is recorded can be set in the drive device 53, and the execution program included in the recording medium 58 is installed in the auxiliary storage device 54 from the recording medium 58 via the drive device 53.

The auxiliary storage device 54 is a storage means such as a hard disk, and can store an execution program according to the present invention, a control program provided in a computer, etc., and perform input / output as necessary.

The memory device 55 stores an execution program or the like read from the auxiliary storage device 54 by the CPU 56. The memory device 55 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.

Based on a control program such as an OS (Operating System) and an execution program read and stored by the memory device 55, the CPU 56 performs various operations and input / output of data with each hardware component, etc. Each process can be realized by controlling the process. Various information and the like can be acquired from the auxiliary storage device 54 during execution of the program, and execution results and the like can also be stored.

The network connection device 57 obtains an execution program from another terminal connected to the communication network by connecting to a communication network or the like, or an execution result obtained by executing the program, or in the present invention. The execution program itself can be provided to other terminals. With the hardware configuration as described above, the person identification process in the present invention can be executed. Also, by installing the program, the person identification process according to the present invention can be easily realized by a general-purpose personal computer or the like. Next, specific contents of the person identification process will be described.

  <Person Identification Process> Next, a person identification process procedure executed by the execution program (person identification program) in the present invention will be described with reference to a flowchart. FIG. 12 is a flowchart illustrating an example of a person identification processing procedure in the present embodiment.

In FIG. 11, first, an image taken by an imaging means such as a camera is input (S01). Next, one or more persons included in the video are detected (S02). Here, when a person is detected, an identification element consisting of a preset feature amount for each person is extracted (S03). Thereafter, in the process of tracking the person, the state of the person is determined (S04), and an unused element is selected from the identification elements obtained in the process of S03 (S05).

Thereafter, person identification is performed (S06), and tracking processing is performed on the photographed person according to the result and instruction (S07). Here, when the tracking pattern or a part of the behavior pattern corresponds to the behavior pattern of the suspicious person set in advance, it is detected as a suspicious person (S08). After the process of S08 is completed, a screen is generated (S09), and the generated screen is output (S10).

Here, it is determined whether or not the tracking is to be ended (S11). When the tracking is not to be ended (NO in S11), the process returns to S04 and the subsequent processing is performed for the same person appearing in the video. If tracking is to be ended (YES in S11), it is determined whether or not the person identification process is to be ended next (S12). If the person identification process is not to be ended (NO in S12), the process of S01 is performed. Returning, another person is detected and the processing described later is performed. If the person identification process is to be terminated (YES in S12), the entire process is terminated.

<Person Identification Processing Procedure: S06> Next, a person identification processing procedure in the above-described processing of S06 will be described with reference to a flowchart. FIG. 13 is a flowchart illustrating an example of a person identification processing procedure. Note that FIG. 13 shows an outline of tracking by detecting the position of the person. In the person identification process shown in FIG. 13, first, it is determined whether or not one or a plurality of persons included in the video (in FIG. 13, persons X1,..., Xn) are detected (S21). If not (NO in S21), the state is updated according to the state transition diagram (S22). Specifically, in the process of S22, for example, “initial state → initial state”, “collation state → initial state”, “tracking state → disappearing state”, “disappearing state → disappearing state or leaving state or initial state”, “ Any setting such as “removal state → initial state” is performed. Here, the “initial state” indicates that the area for storing the tracking information is a free area. For example, when the number of people that can be tracked is three, areas of tracking ID1, ID2, and ID3 are secured, and each holds a tracking state and personal information.

The “collation state” indicates a state in which it is being confirmed whether or not the person is the same as a person detected in the past. In the “verification state”, if it is determined that the person is the same as the new person or the person being tracked, the state shifts to the “tracking state”. Further, in the “collation state”, when it is determined that the region is not stably detected or the region does not move like human (region that seems to be erroneously detected), the process returns to the “initial state”.

The “tracking state” indicates a state in which a person is being tracked. The “tracking state” shifts to the “disappearing state” when the person being tracked cannot be detected. The “disappearance state” indicates a state in which the person being tracked can no longer be detected. The “disappearing state” shifts to the “withdrawal state” after a certain period of time. Further, the “disappearing state” returns to the “tracking state” when the person being tracked is detected again within a predetermined time. Further, the “withdrawal state” indicates a state in which it is determined that the person being tracked is gone. However, when the person returns (when the person who is out of the frame comes into the frame again), the tracking information is held for a certain period of time so that the person can be identified and tracked. In addition, the “withdrawal state” means that after a certain time has elapsed, the information is discarded and the state returns to the “initial state”.

If one or more persons are detected in the process of S21 (YES in S21), it is determined whether or not a person A that is not in an initial state already exists (S23). Here, when there is no person who is not in the initial state (NO in S23), the registration state is updated as “new person to“ initial state → collation state ”(S24). If there is a person who is not in the initial state (YES in S23), it is next determined whether or not the person A is within the movement range (S25). If it is not within the movement range (NO in S25), the registration state is updated as shown in S24 described above.

If it is within the movement range of the person A in the process of S25 (YES in S25), it is next determined whether or not there are a plurality of persons within the movement range (S26). If there are a plurality of persons (YES in S26), person identification processing is performed on person A with persons X1,..., Xn (S27). At this time, the person identification process in S27 is performed using a still image selected from the video. Further, it is determined whether or not there is a person identified in the process of S27 (S28), and if there is no person identified (NO in S28), the registration state is updated as shown in S24 described above.

In addition, if there are not a plurality of persons within the movement range in the process of S26 (NO in S26), or if there is an identified person in the process of S28 (YES in S28), the person A and the person Xi are the same person. The state is updated (S29). This makes it possible to accurately determine whether the fractions included in the video are the same person.

As described above, according to the present invention, the posture of the person can be detected with high accuracy from the process of change for identifying the person with high accuracy according to the situation of the person included in the image.

Specifically, the present invention relates to a method for identifying a person by combining a plurality of ambiguous identification elements (height, body shape, way of walking, gender, gait, etc.) as in processing when a person recognizes a person. Build up. Specifically, in the present invention, an identification element and an identification method are combined. Then, when identifying a person from a certain feature, an optimum feature is selected without using an inappropriate feature. Specifically, the person's state (stopped, walking, whether there was a change in sunshine, etc.) was determined, and various feature values (step length, walking speed, clothes color information, etc.) were determined based on the results. Determine whether to use for identification. The degree of separation is calculated only for the feature amount determined to be used, weighting is performed based on the degree of separation, and the contribution rate for each feature amount to person identification is changed. Thereby, it is possible to perform selection of feature amounts according to the situation.

By applying the present invention, it is possible to recognize, for example, who is the person on the image and what kind of features it is. For example, if “financial institution” is applied to the present invention, designated arrangement information is automatically generated, and if it is a detached house, intrusion at the time of unlocking for a short period of time aiming at garbage disposal is prevented, It is possible to prevent lost children at the facility.

The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10 Person identification apparatus 11 Input means 12 Output means 13 Accumulation means 14 Person detection means 15 Identification element extraction means 16 Identification element selection means 17 Identification means 18 Tracking means 19 Suspicious person detection means 20 Screen generation means 21 Notification means 22 Transmission / reception means 23 Control Means 24 Imaging means 30 Person 31 Image frame 40 Tracking screen 41 Video screen 42 Identification detection screen 43 Tracking progress screen 44 Person area 45 Person estimation information 46 Date and time information 47 Angle of view 48 Current position 49 Tracking route 50 Moving range 51 Input device 52 Output Device 53 Drive device 54 Auxiliary storage device 55 Memory device 56 CPU (Central Processing Unit) 57 Network connection device 58 Recording medium

Claims (12)

In a person identification apparatus for identifying a person included in a video shot at different times, an identification element extraction unit that extracts an identification element of the person, and a shooting that is set in advance from a feature amount obtained by the identification element extraction unit An identification element selecting means for selecting an unused identification element from identification elements already set according to conditions; and an identification means for identifying the person using the identification element obtained by the identification element selecting means. A person identification device characterized by that. The person identifying apparatus according to claim 1, wherein the identification unit resets the weights for the identification elements selected by the identification element selection unit in accordance with a current weighting ratio. The identification element extraction means extracts, as the identification element, a gait, a body type, and gender as feature quantities, and the gait includes a step, a pace, a step speed, a standard deviation of the walking speed, a standard deviation of the walking acceleration, 3. The person identification device according to claim 1, wherein the person identification device has at least one of the inclination and the leg length. When the identification means identifies the person as the same person, the tracking means for tracking the person, the suspicious person detection means for detecting a suspicious person from the tracking result in the tracking means, and the tracking result The person identifying apparatus according to claim 1, further comprising a screen generating unit that generates a screen to be displayed. In a person identification method for identifying a person included in a video shot at different times, an identification element extraction procedure for extracting the identification element of the person, and a shooting set in advance from a feature amount obtained by the identification element extraction procedure An identification element selection procedure for selecting an unused identification element from identification elements already set according to conditions, and an identification procedure for identifying the person using the identification element obtained by the identification element selection procedure A person identification method characterized by that. The person identification method according to claim 4, wherein the identification procedure resets each weighting for the identification element selected by the identification element selection procedure in correspondence with a current weighting ratio. The identification element extraction procedure extracts, as the identification element, gait, body type, and gender as feature quantities, and the gait includes a step, a pace, a step speed, a standard deviation of a walking speed, a standard deviation of a walking acceleration, a body The person identification method according to claim 5, wherein the person identification method has at least one of the inclination and the leg length. When the identification procedure identifies the same person as the person, a tracking procedure for tracking the person, a suspicious person detection procedure for detecting a suspicious person from the tracking result in the tracking procedure, and the tracking result The person identification method according to claim 5, further comprising: a screen generation procedure for generating a screen to be displayed. In a person identification program in a person identification device for identifying a person included in videos taken at different times, a computer, an identification element extraction means for extracting an identification element of the person, and a feature amount obtained by the identification element extraction means The identification element selection means for selecting an unused identification element from the identification elements already set according to the preset photographing conditions, and the identification of the person using the identification element obtained by the identification element selection means A person identification program for functioning as an identification means. The said identification means resets each weighting with respect to the identification element selected by the said identification element selection means corresponding to the ratio of the present weighting, The person identification program of Claim 9 characterized by the above-mentioned. The identification element extraction means extracts, as the identification element, a gait, a body type, and gender as feature quantities, and the gait includes a stride, a pace, a step speed, a standard deviation of a walking speed, a standard deviation of a walking acceleration, a body The person identification program according to claim 8, wherein the person identification program has at least one of the inclination and the leg length. When the identification means identifies the person as the same person, the tracking means for tracking the person, the suspicious person detection means for detecting a suspicious person from the tracking result in the tracking means, and the tracking result The person identification program according to claim 9, further comprising screen generation means for generating a screen to be displayed.