WO2012161291A1

WO2012161291A1 - Site separation location extraction device, program, and method

Info

Publication number: WO2012161291A1
Application number: PCT/JP2012/063403
Authority: WO
Inventors: 康史平川
Original assignee: 日本電気株式会社
Priority date: 2011-05-20
Filing date: 2012-05-18
Publication date: 2012-11-29
Also published as: JP6090162B2; JPWO2012161291A1

Abstract

Provided are a site separation location extraction device, program, and method, with which it is possible to separate and extract a region in an image, such as a person or an object. The site separation location is extracted by: a feature quantity being computed from a person region which is obtained from an image; a value of a person in which the feature quantity is projected in a prescribed axial direction being extracted as a feature change quantity; and the feature change quantity and a feature change quantity for a pre-accumulated known site separation location being compared.

Description

Site separation position extraction apparatus, program, and method

The present invention relates to a part separation position extraction device, a program, and a method.

A monitor or the like can use information such as the color, shape, and type of clothes worn by a person in order to discover a suspicious person from a camera installed for monitoring purposes and express the characteristics of the person. Monitors can also use information such as the color, shape, and type of clothes worn by a person to collect marketing information to understand the trends of customers coming to clothing stores. .
In this field, Patent Document 1 discloses an example of a method for searching for an image showing a person wearing a specific clothes. Patent Document 2 discloses an example of an image search method for searching for an image similar to an input image.

JP 2008-139951 A JP 2007-026386 A

However, the above-described method cannot separate and extract regions such as persons and objects on the image. As a result, the above-described method cannot separate and extract clothes worn by a person from image information.
In order to solve the above problems, an object of the present invention is to provide a part separation position extraction apparatus, program, and method capable of separating and extracting a region such as a person or an object on an image.

To achieve the above object, according to the present invention, for each person area on the image, each line segment in which a plurality of axes parallel to a predetermined axis on the image intersects the person area included in the image. Feature change model accumulating means for storing, as a model, a change amount of a calculated feature amount value with respect to a change in a position on a vertical axis with respect to the predetermined axis, and the change amount for a person region of an external input image. Calculating, comparing the amount of change with the amount of change of the model, extracting a model having a similar amount of change from the feature change model accumulating means, and the value of the amount of change of the extracted model is a predetermined value or more And a part separation position extraction processing means for outputting a peak position which is a part of the part separation position extraction device.
Further, according to the present invention, for each person region on the image, a feature amount calculated for each line segment where a plurality of axes parallel to a predetermined axis on the image and a person region included in the image intersect The change amount is calculated for a human region of the input image from the outside in a computer having a feature change model accumulating unit that stores a change amount of the value of the value with respect to a change in position on the vertical axis with respect to the predetermined axis as a model. The change amount is compared with the change amount stored in the feature change model accumulating means to extract a model having a similar change amount, and the value of the change amount of the extracted model is not less than a predetermined value. Provided is a program storage medium for storing a part separation position extraction program for executing a part separation position extraction processing step for outputting a peak position as a part.
Further, according to the present invention, for each person region on the image, a feature amount calculated for each line segment where a plurality of axes parallel to a predetermined axis on the image and a person region included in the image intersect Is stored as a model with respect to a change in position on the vertical axis with respect to the predetermined axis, the change amount is calculated for a human region of an input image from the outside, and the change amount and the feature change are calculated. A part that compares the amount of change stored in the model storage means, extracts a model having a similar amount of change, and outputs a peak position where the value of the amount of change of the extracted model is a predetermined value or more A separation position extraction method is provided.

According to the present invention, it is possible to provide a part separation position extraction apparatus, program, and method that can separate and extract a region such as a person or an object on an image.

It is a block diagram which shows an example of a structure of 1st Embodiment. It is a flowchart figure which shows an example of operation | movement of 1st Embodiment. It is a block diagram which shows an example of a structure of 2nd Embodiment. It is a flowchart figure which shows an example of operation | movement of 2nd Embodiment. It is a block diagram which shows an example of a structure of 3rd Embodiment. It is a block diagram which shows an example of a structure of 4th Embodiment. It is a block diagram which shows an example of a structure of 5th Embodiment. It is a block diagram which shows an example of a structure of 6th Embodiment. It is a block diagram which shows an example of a structure of 7th Embodiment. It is a flowchart figure which shows an example of operation | movement of 7th Embodiment. It is a figure explaining an example of site | part separation availability determination information. It is a figure which shows an example of the characteristic variation | change_quantity and site | part separation position in the vertical direction of a person area. It is a figure which shows an example of a site | part separation position output process part and a clothing area | region feature extraction process part. It is an example of the association list information stored in the feature change model storage unit. It is an example of the association list information stored in the feature change model storage unit. It is a figure which shows an example of the site | part separation area stored in the horizontal direction characteristic change model storage part 103. FIG. It is a block diagram which shows an example of a structure of 8th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and description thereof will be omitted as appropriate.
Each unit constituting the device of each embodiment includes a control unit, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, a network connection interface, and the like. Realized by a combination of And unless there is particular notice, the realization method and apparatus are not limited.
The control unit includes a CPU (Central Processing Unit) and the like, and controls the entire apparatus by operating an operating system, and reads programs and data from a recording medium mounted on a drive device to a memory, for example. Various processes are executed according to the above.
The recording medium is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, a semiconductor memory, or the like, and records a computer program so that the computer can read it. The computer program may be downloaded from an external computer (not shown) connected to the communication network. Here, unless otherwise specified, the communication network may be the Internet, a LAN (Local Area Network), a public line network, a wireless communication network, or a network configured by a combination thereof.
Each unit constituting the device of each embodiment is configured by a combination of hardware such as a logic circuit, software, and the like. The block diagram used in the description of each embodiment shows a functional unit configuration, not a hardware unit configuration. These functional blocks are realized by any combination of hardware and software. Further, in these drawings, the components of each embodiment may be described as being realized by one physically coupled device, but the means for realizing the same is not limited thereto. That is, the components, devices, systems, and the like described in each embodiment may be realized by a plurality of devices by connecting two or more physically separated devices in a wired or wireless manner. In addition, although there are cases where each component is described as two or more devices that are physically separated, the means for realizing it is not limited to this. That is, each component, device, system, and the like of each embodiment may be realized by arbitrarily combining hardware and software so as to be realized by one physically coupled device.
(First embodiment)
First, a first embodiment will be described with reference to FIG.
FIG. 1 is a block diagram illustrating an example of a site separation position extraction apparatus 1 according to the first embodiment of the present invention. The part separation position extraction apparatus 1 includes a part separation position output processing unit 2. The part separation position output processing unit 2 includes a part separation position extraction processing unit 101 and a feature change model storage unit 102, and the feature change model storage unit 102 is connected to the part separation position extraction processing unit 101.
As shown in FIG. 12, the feature change model accumulating unit 102 has an image feature in a predetermined axial direction defined on an image showing a person, for example, a vertical direction of the person (an axial direction connecting a head and a foot). Change amount (characteristic change amount) of the amount (described later) or the change amount and clothing and body part separation positions (the top of the head, the top of the T-shirt, the bottom of the T-shirt, the top of the skirt, the top of the skirt, etc.) And a feature change model in which a part separation position indicating a part of the body part) is associated. The image feature amount and feature change amount will be described later.
Next, a method in which the part separation position output processing unit 2 stores the feature change model in the feature change model storage unit 102 will be described. The part separation position output processing unit 2 extracts a person area indicating a person area from image information (also simply referred to as an image) input from the outside using an existing method, and determines the size of the person area in the vertical direction. The size of the person area is normalized by enlarging or reducing to a predetermined size. Further, the part separation position output processing unit 2 scans the human area in the horizontal direction, that is, pixel values on a line segment where the axis parallel to the horizontal axis intersects the human area (for example, the type of pixel color) Image feature quantity indicating the feature of the clothing at each position on the axis in the vertical direction of the person is calculated on the basis of the numerical value representing the intensity, brightness, brightness, etc. (see Equation 1 described later). Note that the vertical direction is an axial direction connecting the head and feet of a person in the image, and the part separation position output processing unit 2 may calculate from the image using a known method, or the vertical direction of the camera in advance. The camera is set so that the direction coincides with the vertical direction, and the part separation position output unit 2 may set the vertical direction of the image as the vertical direction.
Then, the part separation position output processing unit 2 calculates a feature change amount indicating a change in the image feature amount that occurs in the vertical direction of the person region, and associates the feature change amount with the type of clothes worn by the person. Information on clothes worn by persons on the image is given in advance.
The part separation position output processing unit 2 obtains each feature change amount for each clothing (see FIG. 12). The part separation position output processing unit 2 uses the peak position of each feature change amount as the part separation position, and uses the information associating the part separation position and the obtained change amount as the above feature change model. Store in the model storage unit 102.
By the way, when the feature change amounts are classified for each type of clothes, the feature change amounts of each classification may be similar. For example, there is a high possibility that the feature change amount obtained from clothes such as sweaters and jeans is similar to the feature change amount obtained from clothes such as Y-shirts and slacks. Therefore, the part separation position output processing unit 2 may combine similar feature change amounts into one. For example, the part separation position output processing unit 2 may combine feature change amounts using an average of each feature change amount when combining similar feature change amounts into one feature change amount. Are substantially the same, one representative feature change amount may be selected from similar feature change amounts based on a predetermined criterion. The part separation position output processing unit 2 may use another method for calculating a feature change amount that is representative of a plurality of feature change amounts.
The part separation position output processing unit 2 uses the degree of similarity calculated using the number of part separation positions and the difference distance between the part separation positions when collecting clothes having similar part separation positions, or DP (Dynamic Programming) matching. The degree of similarity calculated using a method such as the above can be used, but other methods for calculating the degree of similarity may be used.
In addition, the part separation position output processing unit 2 uses the pixel value obtained by scanning the image included in the person region in the horizontal direction as a method for calculating the image feature amount indicating the feature of the person's clothes. The method may be used. That is, the part separation position output processing unit 2 extracts a pixel value of a certain pixel included in the person area, and calculates an image feature amount based on the pixel value and the pixel values of pixels around the pixel. Then, a new image feature amount may be calculated by scanning the image feature amount in the horizontal direction. For example, the part separation position output processing unit 2 considers a plurality of pixels as one pixel instead of one pixel, and scans an average value of pixel values of the plurality of pixels in the horizontal direction to obtain an image feature amount. It may be calculated.
The part separation position output processing unit 2 assigns a clothing name and a body part name (part generic name information) to the section (part separation section) divided by the part separation position, and is characterized as a feature change model. The change model storage unit 102 may store the change model. For example, as shown in FIG. 12, a woman wearing a skirt, stockings, and shoes on a T-shirt has a site separation position at least between the top of the head, the top of the T-shirt, the bottom of the T-shirt, the top of the skirt, the bottom of the skirt, and the shoes. To do. Therefore, the part separation position output processing unit 2 refers to, for example, a storage unit (not shown) in which the section length and the part generic information are stored in association with each other, and each part separation section includes a face, a T-shirt, a skirt, You may give the part generic name information called a leg.
By the way, there are many clothes having the same length of clothes in the vertical direction even if they are different kinds of clothes. For this reason, when similar feature change amounts are combined into one, it is highly likely that feature change amounts of clothes of different types are combined and the same feature change model is calculated. Therefore, the part separation position output processing unit 2 may further add the name of part generic information that summarizes clothes having similar lengths when calculating the feature change amount included in the feature change model. For example, the part separation position output processing unit 2 may use T-shirts, jerseys, blousons, knits and the like as “short upper body clothes (upper body 1)”, and long coats, rain feathers, etc. as “long upper body”. Clothing (upper body 2) ”, mini skirts, shorts and half pants“ short lower body clothing (lower body 1) ”, lower body clothing such as seven-part trousers and long skirts that can be seen near the ankle, etc. “Lower body clothes (lower body 2)”, long pants that cannot see the ankles, long skirts, and the like may be classified as “long lower body clothes (lower body 3)”. Further, the part separation position output processing unit 2 may add not only the part separation section but also part generic name information about the entire clothing to the feature change model. For example, a combination of a jacket and slacks may be given part generic information “suits”.
By the way, depending on clothes, the gender that can wear the clothes may be determined. Therefore, the part separation position output processing unit 2 may add information (partial sex information) such as male, female, and unisex that is determined for each clothing to the feature change model. The part separation position output processing unit 2 may also store this part sex information in the feature change model storage unit 102. For example, since the person wearing the skirt is generally a woman, the part separation position output processing unit 2 refers to, for example, a storage unit (not shown) in which part generic information and part sex information are stored in association with each other. It is possible to give a woman as part sex information to the skirt.
The part separation position output processing unit 2 may further add attribute information such as a person's sex, age, height, race, hairstyle (such as length) to the feature change model together with the part sex information. For example, the attribute information includes “male / female” for gender, “length” and “style” for hairstyle, “age group” for age, and “skin color” for race. The part separation position output processing unit 2 can extract the attribute information from a person shown in the image using a known technique.
The part separation position output processing unit 2 may also store these attribute information in the feature change model storage unit 102. In addition to the information described above, the part separation position output processing unit 2 can add to the feature change model if there is other attribute information and can store it in the feature change model storage unit 102.
The accumulated part sex information, part generic name information, and attribute information can be used for narrowing down feature change models extracted from the feature change model storage unit 102 when matching feature change models to be described later. The part sex information, the part generic name information, and the attribute information are collectively referred to as a person attribute.
The part sex information, part generic information, attribute information, part separation section length (part separation section length), ID (IDentification) for indicating a correspondence relationship between feature change amounts, and the like are, for example, an association list as shown in FIG. Information may be stored in the feature change model storage unit 102 as information. The saved format may be another data format.
The feature change model storage unit 102 may store the feature change model for each direction of the person. The part separation position output processing unit 2 detects, for example, that a person is facing in the horizontal direction or the rear direction by a known technique, calculates a feature change amount and separation position information for each person's direction, and models each direction. May be stored in the feature change model storage unit 102. For example, the feature change model accumulating unit 102 stores feature change models constructed for the front direction, the horizontal direction, and the rear direction for suits that look different from the front direction, the horizontal direction, and the rear direction. You may accumulate according to direction.
Note that the above-described process has been described as the process in which the part separation position output processing unit 2 stores the feature change model in the feature change model storage unit 102, but is not limited thereto. For example, a feature change model generated in advance by an external device (not shown) or the like may be stored in the feature change model storage unit 102.
As described above, the feature change model storage unit 102 has the feature change amount of the image feature amount or the feature change amount and the part separation position at each position on the vertical axis of the person as shown in FIG. The associated feature change model is accumulated. In addition, as described above, the feature change model storage unit 102 may further store person attributes (partial sex information, part generic name information, attribute information) and part separation section length. Further, the feature change model storage unit 102 may store a probability distribution indicating the frequency of occurrence of a location where the feature change amount of the image feature amount on the vertical axis of the person is equal to or greater than a threshold value.
The part separation position extraction processing unit 101 determines whether the image information, the person area information (information indicating the position of the area in which the person is shown in the image information), and the clothes of the person area can be separated. Using the part separation possibility determination information to be determined, the feature change amount of the person region is calculated as shown in the right graph of FIG. 12, and the feature change amount and the feature change accumulated in the feature change model storage unit 102 are calculated. The similarity with the feature change amount of the model is calculated. The part separation position extraction processing unit 101 can calculate the degree of similarity using a method such as DP (Dynamic Programming) matching, using the number of part separation positions and the difference distance between the part separation positions. Other similarity calculation methods may be used.
The part separation possibility determination information is information indicating whether or not a person can be separated according to how the person in the image is captured. In the part separation possibility determination information, a value that determines whether separation is possible is described. For example, the part separation possibility determination information is a binary value indicating whether or not it is possible, or a numerical value for performing threshold processing. The part separation availability information is calculated by an external device (not shown). For example, the external device can determine how to capture a person showing that the person area is shown in the whole body in the image or the lower body is partially hidden, and can determine whether or not the part can be separated based on the way the image is taken. A specific example of the part separation availability information will be described with reference to FIG. The left figure of FIG. 11 is an input image, and shows a situation where only three upper persons can be seen because there are three persons and one person stands at the back of the desk. In this case, the external device can calculate a region as shown in the right diagram of FIG. 11 by a known method based on, for example, the position of the person's feet. In the right figure of FIG. 11, the upper end of the desk is an area where only the upper body is shown (black area), the floor is an area where the whole body is shown (shaded area), and the wall is an area where a person's feet cannot exist (white area) ). As described above, the part separation possibility determination information is “Yes” if there is a region in the image where there is a possibility of part separation, and “No” if there is no region.
The part separation position extraction processing unit 101 outputs the part separation position of the input person region based on the part separation position of the feature change model having the highest feature change amount. At this time, the part separation position extraction processing unit 101 can also receive and output information such as part generic name information, part sex information, attribute information, and part separation section length from the feature change model storage unit 102. For example, the part separation position extraction processing unit 101, when the feature change amount of clothes worn by a person in the input image is similar to a plurality of feature change amounts stored in the feature change model storage unit 102, Site generic information can be output. For example, since it is difficult to distinguish between a person wearing a T-shirt and a person wearing a knit or a trainer by a part separation position, the part separation position extraction processing unit 101 collectively refers to the upper body as “short upper body” The part generic information such as “clothing (upper body 1)” can be output.
Next, the operation of the first embodiment will be described in detail with reference to FIG.
The part separation position extraction processing unit 101 determines whether or not part separation is possible from the person region on the image based on the part separation possibility determination information (step 1001). The part separation position extraction processing unit 101 determines whether or not separation is possible based on the part separation possibility determination information.
When the part separation possibility determination is possible (step 1001, yes), the part separation position extraction processing unit 101 normalizes the person region (step 1002). The reason for performing the normalization is to suppress the variation of the feature change amount due to the change in the person area size. As a specific normalization method, the part separation position extraction processing unit 101 performs an enlargement or reduction process of an image while maintaining a vertical / horizontal aspect ratio so that the lengths of the human regions in the vertical direction are aligned with the reference value. The enlargement / reduction processing method may be a general image enlargement / reduction algorithm, such as the nearest neighbor method, the bilinear method, the bicubic method, or the like.
After normalization, the part separation position extraction processing unit 101 calculates an image feature amount in the horizontal direction of the person region. The part separation position extraction processing unit 101 scans pixel values of an image in the range indicated by the human region information in the horizontal direction (x-axis direction) and uses a function such as a pixel value projected in the vertical direction (y-axis direction). It is obtained and used as an image feature amount (step 1003). Specifically, the part separation position extraction processing unit 101 obtains an image feature amount on a line segment where the person region and the x axis intersect, for example, as in (Equation 1).

Here, I (x, y) is a pixel value at coordinates (x, y) on the input image (for example, a value calculated from a three-dimensional vector of R (Red), G (Green), and B (Blue)). , M (x, y) is mask information generated from the person area information, X0 is the minimum value of the x coordinate in the person area, and X1 is the maximum value of the x coordinate in the person area.
Next, the part separation position extraction processing unit 101 calculates the change amount (feature change amount) from the image feature amount. That is, the part separation position extraction processing unit 101 calculates the feature change amount of the image feature amount projected in the vertical direction (step 1004). For example, the part separation position extraction processing unit 101 determines the position of the image feature value calculated for each line segment where a plurality of axes parallel to the horizontal axis intersect with the person area on the vertical axis. A feature change amount with respect to the change is calculated. The feature change amount is obtained as in (Equation 2), for example. B is an upper limit of the range of the image feature amount used for obtaining the feature change amount.

The part separation position extraction processing unit 101 calculates the similarity between the calculated feature change amount and each feature change amount included in the feature change model stored in the feature change model storage unit 102 in advance. A feature change model having a feature change amount is extracted, and a peak position (part separation position) of the feature change amount of the model is output (step 1005). For example, the part separation position extraction processing unit 101 outputs the part separation position of the feature change amount having the highest similarity.
The part separation position extraction processing unit 101 extracts, for example, a value in which D (y) in (Equation 2) is a peak equal to or greater than a threshold as a method for calculating the similarity, and the position of the peak is determined as the part separation position. Let it be a candidate (part separation candidate position). The part separation position extraction processing unit 101 then determines the number of part separation positions and the difference distance between the part separation positions based on the part separation positions accumulated in the feature change model accumulation unit 102 and the extracted part separation candidate positions. Or the similarity of the feature change amount can be calculated by using a technique such as DP matching.
Further, the part separation position extraction processing unit 101 obtains a product of the feature change amount of (Equation 2) and the feature change amount of the feature change model stored in the feature change model storage unit 102, and uses the product distribution. The number of peaks that are greater than or equal to the threshold value and the height of the peak may be detected, and the similarity of the feature change amount may be determined based on the number of peaks and the peak height. The part separation position extraction processing unit 101 may determine the similarity of the feature change amount using other methods.
According to the present embodiment, by outputting the peak position of a feature change model in which the change amount of the image feature quantity is similar to the clothes worn by the person on the input image, the parts of the clothes are separated. And can be extracted. The feature change amount and its peak position extracted from the input image may not always accurately indicate the separation position of the part due to image noise or the like. Even in such a case, according to the present embodiment, the part separation position extraction processing unit 101 is obtained from the input image using the feature change model whose part separation position is already known. Since the feature change amount similar to the feature change amount is specified and the part separation position of the feature change amount is output, an accurate part separation position can be output.
Further, the part separation position output in this way is output from the part separation position output processing unit 2 and input to the clothing region feature extraction processing unit 3 that extracts the clothing region feature amount, for example, as shown in FIG. May be. By extracting the part separation position according to the present embodiment, the clothing region feature extraction processing unit 3 causes the image information input to the clothing region feature extraction processing unit 3, person region information indicating a person region, and the part separation position. It is possible to obtain a clothing region for each part using, and to extract a feature amount of the clothing region using pixel information included in the clothing region. As the feature amount of the clothing region, for example, a histogram indicating colors / edges (patterns) from the clothing region may be used, or feature amounts indicating other colors / patterns may be used. The feature amount enables searching for a specific person, searching for a person using a clothing area, and the like.
Further, although the present embodiment has been described with respect to the extraction of the part separation position in the clothes of the person, according to the present embodiment, the part separation is not limited to the person or the clothes, but also to animals and objects in the same manner as described above. It is possible to extract the position.
(Second Embodiment)
Next, a second embodiment of the present invention will be described.
FIG. 3 is a block diagram showing an example of another configuration of the part separation position output processing unit 2 shown in FIG.
The part separation position output processing unit 2 in the present embodiment includes a part separation position extraction processing unit 107, a feature change model storage unit 102, a person specifying information extraction / collation processing unit 301, and a person attribute storage unit 201. Including.
The feature change model accumulation unit 102 accumulates not only the feature change amount or the part separation position but also the above-described personal attributes (part generic name information, part sex information, attribute information) and the part separation section. That is, the feature change model stores the feature change model and the person attribute in association with each other.
In the person attribute storage unit 201, information for specifying a person, for example, a person individual feature amount used for specifying a person by face recognition, and a person attribute (part sex information, part generic information, attribute information) are associated with each other. Accumulated. As described above, the part generic information is a part generic name that summarizes similar clothes when calculating a probability distribution of a part separation position included in a feature change model (T-shirt, skirt, etc.) and a feature change model. Information name (upper body clothes (upper body 1), etc.), part generic information (suits) considering the entire clothes. As described above, the person attribute storage unit 201 is given a name with the same granularity as the part generic name information stored in the feature change model storage unit 102.
The person identification information extraction / collation processing unit 301 extracts a person's face area based on the input image information and person area information, and uses the existing method to identify the individual individual feature amount used for face collation from the face area. Is calculated. The person specifying information extraction / collation processing unit 301 compares this individual individual feature amount with the individual individual feature amount stored in the individual attribute storage unit 201, and determines the person attribute corresponding to the most similar individual individual feature amount as the person. Extracted from the attribute storage unit 201 and output to the part separation position extraction processing unit 107.
Various techniques have been proposed for the face area extraction technique and the face recognition technique, but the technique used by the person specifying information extraction / collation processing unit 301 is not particularly limited. The individual individual feature amount is not limited to the feature amount extracted from the face region, and may be a feature amount such as a person's body shape, gait feature, iris information, fingerprint information, ID information assigned to each person. Further, the person identification information extraction / collation processing unit 301 may have a determination function such as gender, age, height, race, hairstyle, and the like. Accordingly, the person specifying information extraction / collation processing unit 301 may obtain the individual individual feature amount based on information such as sex, age, height, race, and hairstyle extracted from the person region.
The part separation position extraction processing unit 107 has a function of extracting a part separation position similar to the function of the part separation position extraction processing unit 101 of FIG. 1 described in the first embodiment.
The part separation position processing unit 107 acquires the person attribute stored in the person attribute storage unit 201 from the person specifying information extraction / collation processing unit 301 and extracts a person attribute similar to the person attribute from the feature change model storage unit 102. To do. Next, the part separation position processing unit 107 extracts a feature change amount corresponding to a person attribute similar to the extracted person attribute from the feature change model storage unit 102. Accordingly, the part separation position extraction processing unit 107 can narrow down the feature change amount stored in the feature change model storage unit 102. The part separation position extraction processing unit 107 improves collation accuracy and processing speed compared to the part separation position information extraction unit 101 of FIG.
Next, the operation of the second embodiment will be described in detail with reference to FIG.
From the part separation possibility determination process in step 1001 to the calculation of the feature change amount by projection of the feature change amount in the person vertical direction in step 1004, the part separation position extraction processing unit 107 performs the same process as in FIG.
The person identification information extraction / collation processing unit 301 collates the individual individual feature amount extracted from the person region with the individual individual feature amount accumulated in the person attribute accumulation unit 201, and is similar to the person attribute accumulation unit 201. A person attribute associated with a person individual feature amount is extracted. The person specifying information extraction / collation processing unit 301 outputs the extracted person attribute to the part separation position extraction processing unit 107.
The part separation position extraction processing unit 107 collates the received person attribute with the person attribute included in the feature change model stored in the feature change model storage unit 102, and calculates the feature change amount associated with the similar person attribute. Extract. That is, the part separation position extraction processing unit 107 narrows down the feature change model (step 1006).
Next, the part separation position extraction processing unit 107 performs the same processing as step 1005 in FIG. 2 at step 1005 in FIG. 4 and the feature change amount of the feature change model extracted and narrowed down at step 1006 and step 1004. Is compared with the feature change amount obtained in step 1, and the part separation position of the most similar feature change amount is output.
According to the present embodiment, the collation accuracy and the processing speed are improved as compared with the first embodiment. This is because the part separation position extraction processing unit 107 narrows down the feature change model used for collation.
(Third embodiment)
FIG. 5 is a block diagram illustrating an example of another configuration of the part separation position output processing unit 2 of FIG.
The part separation position output processing unit 2 in the present embodiment includes a part separation position extraction processing unit 105, a feature change model storage unit 102, and a horizontal direction feature change model storage unit 103.
The site separation position extraction processing unit 105 includes a vertical direction feature change amount matching processing unit 210, a horizontal direction feature change amount matching processing unit 211, and a site separation position integration processing unit 212.
The feature change model storage unit 102 is connected to the vertical feature change amount matching processing unit 210. The horizontal feature change model storage unit 103 is connected to the horizontal feature change amount matching processing unit 211. The part separation position integration processing unit 212 is connected to the vertical direction feature change amount matching processing unit 210 and the horizontal direction feature change amount matching processing unit 211. The feature change model storage unit 102 is the same as that in FIG.
Like the feature change model storage unit 102, the horizontal feature change model storage unit 103 stores feature change amounts with known part separation positions. However, the feature change amounts are not the vertical direction of the person but the horizontal direction. Extracted with respect to direction. In other words, the horizontal direction feature change model accumulation unit 103 scans from the pixel value of the image of the person area in the vertical direction (y-axis direction) of the person, and calculates from the pixel value projected onto the axis in the horizontal direction (x-axis direction). The feature change amount (horizontal feature change amount) is stored.
The horizontal direction feature change model accumulating unit 103 includes, in addition to the horizontal direction feature change amount, a part separation section, a person attribute (part sex information, part generic information, attribute information), etc., in the same manner as the feature change amount calculation in the vertical direction. You may save it. At that time, the horizontal direction feature change model accumulating unit 103 may store such information in a list format as shown in FIG.
As an example of the part separation section stored in the horizontal direction feature change model storage unit 103, for example, an image of a person wearing a cardigan on a T-shirt and wearing jeans (assuming that the cardigan is open before the cardigan) (See FIG. 15). In this case, the cardigan fabric is sandwiched between the T-shirt fabric from the left and right, and the jeans are located at the lower end of the T-shirt. In this case, as an example, part generic information such as cardigan, T-shirt, jeans, T-shirt, and cardigan is given to the part separation section. However, since there are many clothes having the same length in the horizontal direction, there is a high possibility that the horizontal feature change amounts are similar. Therefore, similarly to the description of the feature change amount in the vertical direction, the part separation position output processing unit 2 assigns part generic information including clothes having similar horizontal feature change amounts to the feature change model. Also good. Furthermore, the part separation position output processing unit 2 may add not only the part separation section but also part generic name information considering the entire clothing to the feature change model. The process of giving the part generic information as described above is the same as the method described in the feature change model in the vertical direction of the first embodiment.
Similarly to the first embodiment, depending on the clothes, the gender on which the clothes can be worn may be determined. Therefore, the site separation position output processing unit 2 is determined by men and women determined for each clothes. In addition, it is also possible to assign part sex information such as unisex to the horizontal feature change model. The horizontal direction feature change model accumulating unit 103 may store this part sex information. The part separation position output processing unit 2 can add part sex information of unisex to the feature change model of the horizontal direction feature change model storage unit 103 from clothes information such as jeans, T-shirts, and cardigans.
Furthermore, as in the first embodiment, the part separation position output processing unit 2 may also add attribute information such as age, height, race, hairstyle (style, length, etc.) to the feature change model. . In addition, the part separation position output processing unit 2 may add to the feature change model if there is attribute information other than the information described above. In addition, as described above, the horizontal direction feature change model storage unit 103 includes the part sex information, the part generic information, the attribute information, the part separation section length, the ID for indicating the correspondence between the feature change amounts, the part separation position, and the clothes. You may save the information etc. which linked | related with the name of.
Although the above-described processing has been described as processing performed by the part separation position output processing unit 2, a result processed in advance by an external device or the like may be stored in the horizontal direction feature change model accumulation unit 103.
Similar to the part separation position extraction processing unit 101 described based on FIG. 1, the vertical feature change amount matching processing unit 210 obtains image information, person area information, and part separation possibility determination information from the outside. Then, when it is determined that the part separation is possible from the part separation possibility determination information, the vertical direction feature change amount matching processing unit 210 extracts the feature change amount based on the input image information and person area information. . Then, the vertical feature change amount matching processing unit 210 calculates the similarity between the feature change amount and the feature change amount of the feature change model stored in the feature change model storage unit 102, and has the highest similarity. The part separation position on the vertical axis is output.
Similar to the vertical direction feature change amount matching processing unit 210, the horizontal direction feature change amount matching processing unit 211 acquires image information, person area information, and part separation possibility determination information from the outside. When it is determined that the part separation is possible from the part separation possibility determination information, the horizontal direction feature change amount matching processing unit 211 extracts the part separation position. However, the horizontal feature change amount matching processing unit 211 extracts a part separation position in the horizontal direction of the person, not in the vertical direction of the person. Specifically, the horizontal direction feature change amount matching processing unit 211 calculates the horizontal direction feature change amount, and the calculated horizontal direction feature change amount and the horizontal direction feature change amount extracted from the horizontal direction feature change model storage unit 103. And the part separation position of the horizontal feature change model having the highest horizontal feature change amount is output.
The part separation position integration processing unit 212 acquires the vertical part separation position extracted by the vertical feature change amount matching processing unit 210 and the horizontal part separation position extracted by the horizontal direction feature change amount matching processing unit 211. Then, those pieces of information are integrated to output the part separation position of the entire person.
As the part separation position integration processing, the part separation position integration processing unit 212 generates information for dividing the person region by combining the part separation positions in each direction. For example, the part separation position integration processing unit 212 extracts the positions of the head, upper body, and lower body areas from the part separation section, the part generic information, and the person area that are stored as the part separation positions in the vertical direction, and the horizontal direction Using this part separation position, an arm region (a clothing region when layered) that is a part of the upper body can be extracted from the upper body region of the person.
According to the present embodiment, the part separation position integration processing unit 212 determines the position of each part region in the horizontal and vertical directions, and thus can extract more detailed part separation positions. Further, the part separation position integration processing unit 212 can increase the accuracy of the clothes name and part sex information of the person region by using the part separation positions in the horizontal and vertical directions. For example, the common part generic name included in the part separation positions in each direction is highly likely to be clothes worn by the person region. In addition, the part separation position integration processing unit 212 can be said to have high accuracy when the gender included in the part gender information in each direction is common, and low accuracy when the gender is different. As described above, the part separation position integration processing unit 212 can output part separation information in which clothing names, part sex information, and the like are integrated for each direction.
(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.
FIG. 6 is a block diagram illustrating an example of another configuration of the part separation position extraction processing unit 105 illustrated in FIG.
In order to extract the part separation position from the person region, in the third embodiment (see FIG. 5), the vertical direction feature change amount matching processing unit 210 and the horizontal direction feature change amount matching processing unit 211 are arranged in parallel. A feature change amount matching process was performed. On the other hand, in FIG. 6, the vertical direction feature change matching processing unit 210 and the horizontal direction feature change matching processing 213 are connected in series. The purpose of connecting in series is that the horizontal feature change amount matching process 213 extracts a horizontal part separation position for each part divided in the vertical direction extracted by the vertical direction feature change amount matching processing unit 210. It is. Cases where the number of occurrences of feature changes in the vertical direction of a person is greater than the number of occurrences of feature changes in the horizontal direction of the person, so that the part separation position in the horizontal direction is more difficult to extract than the part separation position in the vertical direction Can occur. For example, when calculating the horizontal part separation position in the head, legs, front-opening clothes, etc., the horizontal part separation position tends to concentrate on the same position. Therefore, it is difficult to distinguish and extract part separation positions such as the head, legs, and front-opening clothes in the horizontal direction. Therefore, the part separation position extraction processing unit 105 first specifies the part separation position in the vertical direction as in the configuration of FIG. 6, and extracts the horizontal feature change amount for each part in the vertical direction, thereby achieving higher accuracy. The site separation position can be extracted.
At this time, it is not necessary to extract the horizontal direction feature change amount for all the parts in the vertical direction. For example, the part separation position extraction processing unit 105 may perform the horizontal feature change amount extraction when the upper body of the person is layered and the lower body is pants. On the other hand, the part separation position extraction processing unit 105 may not calculate the horizontal feature change amount for an area that does not need to be further divided, such as a head part or a shoe part.
The site separation position extraction processing unit 105 in the present embodiment includes a vertical direction feature change amount matching processing unit 210, a horizontal direction feature change amount matching processing unit 213, and a site separation position integration processing unit 212.
The part separation position integration processing unit 212 is connected to the vertical direction feature variation matching processing unit 210 and the horizontal direction feature variation matching processing unit 213. The part separation position integration processing unit 212 includes information such as the vertical part separation position of the vertical feature change amount matching processing unit 210 and information such as the horizontal part separation position of the horizontal feature change amount matching processing unit 213. To generate and output information such as part separation positions in the entire person area.
The horizontal direction feature amount matching processing unit 213 is connected to the vertical direction feature amount matching processing unit 210 and the part separation position integration processing unit 212. Image information, person region information, and a vertical feature change model are input to the vertical feature change amount matching processing unit 210 from the outside. Also, the horizontal part separation position generated by the vertical direction feature variation matching processor 210 is input to the horizontal feature variation matching processor 213.
The horizontal feature change amount matching processing unit 213 extracts a horizontal feature change amount for each vertical part separation section based on the vertical part separation position, and the horizontal feature change amount and the input horizontal feature change amount are extracted. The horizontal feature change model included in the direction feature change model is collated, and a horizontal feature change model including a similar feature change amount is searched from the horizontal feature change model storage unit 103.
The horizontal feature change amount matching processing unit 213 outputs the horizontal part separation position of the feature change model including the similar feature change amount to the part separation position integration processing unit 212.
The part separation position integration processing unit 212 integrates the vertical part separation position of the vertical direction feature change amount matching processing unit 210 and the horizontal part separation position of the horizontal direction feature change amount matching processing unit 213, and Generate and output site separation information.
According to the present embodiment, first, based on the vertical part separation position of the person, specify the vertical part of the person, and then extract the horizontal feature change amount for each vertical part, A site separation position with higher accuracy can be extracted.
(Fifth embodiment)
FIG. 7 is a block diagram illustrating an example of another configuration of the part separation position extraction processing unit 101 illustrated in FIG. 1 according to the first embodiment.
The part separation position extraction processing unit 101 of this embodiment includes a person vertical direction extraction processing unit 207 and a feature change amount matching processing unit 206, and the feature change amount matching processing unit 206 is connected to a feature change model storage unit 102 (not shown). Has been. The feature change model storage unit 102 is the same as that shown in FIG.
The person vertical direction extraction processing unit 207 receives image information and person area information from the outside. The person vertical direction extraction processing unit 207 estimates the vertical direction of the person area from the input image information and person area information. There are various methods for estimating the vertical direction. For example, a head detector or a foot detector generated by learning in advance is used to detect the head and feet, and a method for estimating the vertical direction from a vector connecting the two points of the head and feet, or represents the head or feet 2D coordinates on the image are converted into 3D space coordinates using the camera calibration information, and 3D space coordinates of a point on a straight line passing through the point and parallel to the vertical direction are obtained and converted back to 2D. There are methods. Specifically, when the lower end of the person area is a foot, the person vertical direction extraction processing unit 207 converts the image coordinates from two-dimensional to three-dimensional based on the position information of the space to be photographed and information such as camera parameters. Convert the foot position of the two-dimensional coordinates to the foot position of the three-dimensional coordinates by using a transformation matrix that projects in reverse, and set the three-dimensional coordinates set with an appropriate height from the foot positions and reversely transform to two dimensions. Thus, the vertical direction of the person on the two-dimensional space, that is, the image can be extracted. As a result, the vertical direction of the person can be automatically set even in a region where the y-axis and the vertical direction do not coincide with each other, such as a region around the lens, due to lens distortion or the angle of view. The separation position can be acquired.
The feature change amount matching processing unit 206 has a function of extracting a part separation position in the same manner as the part separation position extraction processing unit 101 in FIG. Similarly to the first embodiment, the feature change amount matching processing unit 206 performs the first embodiment based on the information in the vertical direction estimated for each person area acquired from the person vertical direction extraction processing unit 207. Similar to the form, the image feature amount of the person region is calculated.
According to the present embodiment, the image feature amount in the vertical direction can be calculated based on the information on the vertical direction of the person area estimated by the person vertical direction extraction processing unit 207, so that the camera position is manually set. The image feature amount in the vertical direction can be calculated without setting.
(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described.
FIG. 8 is a block diagram illustrating an example of another configuration of the part separation position extraction processing unit 101 illustrated in FIG. 1 according to the first embodiment.
The part separation position extraction processing unit 101 in this embodiment includes a person orientation determination unit 209 and a feature change amount matching processing unit 208, and the feature change amount matching processing unit 208 is connected to a feature change model storage unit 102 (not shown). ing. The person orientation determination unit 209 is connected to the feature change amount matching processing unit 208.
A feature change model storage unit 102 (not shown) stores feature change models for each direction of a person. The direction of a person (front, back, sideways, etc.) is expressed by an angle, a vector, or the like based on a specific direction. For example, a coordinate system is defined on the basis of the direction in which the person is facing the front toward the camera, and the direction of the person is expressed by an angle or a vector direction. These angles and vector information are defined as person orientation information. The feature change model storage unit 102 stores feature change models classified according to person orientation information.
The person orientation determination unit 209 determines the direction in which the person is facing based on the image information and the person area information, and outputs the person orientation information. There are various methods for determining the direction of a person, but the direction of the face that the person faces by face recognition is set to the direction that the person is facing, or the direction of travel by optical flow using images input in time series There are a method for estimating the direction of a person from the above, a method for calculating the position change of the extracted person region, and a method for estimating the direction of the person from the traveling direction. In the second and third methods, the person orientation determination unit 209 needs to know the camera coordinate system. For example, the person orientation determination unit 209 is configured to display the rear side when the person moves away from a straight line with respect to the camera orientation. On the other hand, the direction of the person can be estimated as a front direction when approaching linearly and as a landscape direction when crossing.
The feature change amount matching processing unit 208 has a function of extracting a part separation position in the same manner as the part separation position extraction processing unit 101 in FIG. In addition, the feature change amount matching processing unit 208 compares the feature change amount for the input image with the feature change amount of the feature change model stored in the feature change model storage unit 102 before the person orientation determination unit 209. The person orientation information obtained from the above and the person orientation information that classifies the feature change model are collated. As a matching method, the feature change amount matching processing unit 208 compares, for example, the angle of person orientation information and the similarity between vectors. For example, when the person orientation information is an angle, the feature change amount matching processing unit 208 may compare the similarity based on the difference between the angles, and may compare the similarity based on an inner product if the information is a vector. However, other methods for comparing similarities may be used.
Thereafter, the feature change amount matching processing unit 208 selects the person orientation information with the most similar person orientation information, selects the feature change model corresponding to the person orientation information, and compares the feature change amounts. Accordingly, the feature change amount matching processing unit 208 can perform matching between feature change amounts of the same or similar person orientation.
According to the above configuration, first, a feature change model with a similar person orientation is extracted and narrowed down, and feature change amounts are compared with each other in the feature change model. Compared to the configuration of FIG. 1, the site separation position can be acquired with higher accuracy with respect to fluctuations in the appearance of clothes.
(Seventh embodiment)
FIG. 9 is a block diagram illustrating an example of another configuration of the part separation position extraction processing unit 101 illustrated in FIG. 1 according to the first embodiment.
The part separation position extraction processing unit 101 in the present embodiment includes a part separation position accumulation unit 203, a person region information accumulation unit 204, and a feature change amount matching processing unit 205, and the feature change amount matching processing unit 205 is not illustrated. The feature change model storage unit 102 is connected.
The part separation position accumulating unit 203 accumulates the feature change amount (or part separation position) calculated by the feature change amount matching processing unit 205 together with time stamp information at the time of image information acquisition. The part separation position accumulating unit 203 outputs the past feature change amount (or part separation position) and time stamp information that have already been accumulated to the feature change amount collation processing unit 205.
The person area information accumulation unit 204 accumulates time stamp information at the time of image information acquisition and person area information.
In addition, the person area information storage unit 204 outputs the past person area information that has already been stored to the feature change amount matching processing unit 205 together with the time stamp information.
The feature change amount matching processing unit 205 receives externally input image information to which time stamp information is added, person region information, and part separation possibility determination information. Further, the feature change amount matching processing unit 205 receives a feature change model from a feature change model storage unit 102 (not shown).
In addition, the feature change amount matching processing unit 205 is close to the time stamp information of the input image information (the time indicated by the time stamp information and a time within a predetermined time), and the position and size of the person area are close ( In order to collate a past person area (with an error within a predetermined range), it is connected to the person area information storage unit 204. Further, the feature change amount matching processing unit 205 is connected to the part separation position accumulating part 203 in order to extract a part separation position at a time close to the time stamp information.
When the person region information storage unit 204 can extract past person regions at a time close to the time stamp information, the feature change amount matching processing unit 205 can extract the past person regions at a time close to the time stamp information. It may be regarded as a person area of the same person. This is because it can be assumed that a person who appears in an image that has not passed much time will not change its position or size abruptly. In addition, as a method for determining whether or not the two person regions are the same person, the feature change amount matching processing unit 205 calculates an optical flow that calculates the change in the luminance information of the image and estimates the movement amount of the pixel. It may be used. In this case, the feature change amount matching processing unit 205 can regard a person area with a small amount of movement as a person area of the same person. Further, the feature change amount matching processing unit 205 can also determine whether or not the person area is the same person by extracting the pixel value of the person area range from the image information and looking at the same color and luminance. . However, in this case, the person area information storage unit 204 stores not only the person area information but also the image information together with the time stamp information.
The feature change amount matching processing unit 205 extracts a person area regarded as the person area of the same person from the person area information storage unit 204, and time stamp information at a time close to the time stamp information given to the person area information. The amount of feature change (or part separation position) to which is added is extracted from the part separation position accumulation unit 203. This feature change amount (or part separation position) is considered to represent the feature change amount (or part separation position) of the same person.
The feature change amount matching processing unit 205 extracts the feature change amount extracted from the extracted feature change amount (or part separation position) and the image information and person area information currently input to the feature change amount matching processing unit 205. Alternatively, the similarity with the part separation candidate position calculated in the same manner as in the first embodiment can be calculated, and the most similar part separation position can be extracted.
On the other hand, when there is no person area that can be regarded as a person area of the same person from the person area information storage section 204, that is, the position and size of the person area information is stored in the person area information storage section 204. When only a significantly different person region exists, the feature change amount matching processing unit 205 outputs the part separation position by the same processing as in the first embodiment.
When a plurality of part separation positions of the same person are stored in the part separation position storage unit 203, the part separation position extraction processing unit 101 uses the average position of the plurality of part separation positions or the time stamp is new. A part separation position can be created using a weighted average position that gives a greater weight to the part separation position. In addition, the part separation position extraction processing unit 101 can create a part separation position by obtaining a probability distribution (for example, a Gaussian distribution) indicating the frequency of the part separation positions extracted from a plurality of images. Thereby, the part separation position caused by the noise of the image information can be removed, and the matching accuracy of the part separation position is improved. In this case, in addition to the part separation position, the person region information storage unit 204 stores an ID for specifying a person in association with the part separation position.
Next, the operation of each part of the present embodiment will be described in detail. FIG. 10 is a flowchart showing an example of the operation of the part separation position extraction processing unit 101.
From the part separability determination process in step 1001 to the calculation of the feature change amount by the projection of the feature change amount in the human vertical direction in step 1004, the same process as that in FIG.
Next, the feature change amount collation processing unit 205 extracts a person area to which a time stamp that is similar to the time stamp given to the input image is given from the person area information storage unit 204, and determines the size and position of the person area. From this, it is determined whether or not the same person area (step 1007). When it is determined in step 1007 that the person area is the same person (step 1007, Yes), the feature change amount matching processing unit 205 is given time stamp information at a time close to the time stamp information from the part separation position accumulating unit 203. The amount of feature change (or part separation position) is extracted. The feature change amount matching processing unit 205 obtains a feature change amount based on the image information input to the feature change amount matching processing unit 205 and the person area (or described in step 1005 of the first embodiment). The part separation candidate position is extracted from the feature change amount by the technique). The feature change amount matching processing unit 205 determines the similarity between the feature change amount (or part separation candidate position) and the feature change amount (or part separation position) extracted from the part separation position storage unit 203 in the first embodiment. Calculation is performed by the method described in the embodiment, and a part separation position is extracted (step 1008).
On the other hand, when it is not determined that the person area is the same person (No in Step 1007), the feature change amount matching processing unit 205 outputs the part separation position by the same process as in the first embodiment.
According to the above configuration, since the part separation position is acquired using the person region of the same person in the past, the part separation position can be obtained with high accuracy.
(Eighth embodiment)
The part separation position extraction apparatus 1 according to the present embodiment includes a feature change model accumulation unit 102 and a part separation position extraction processing unit 101, as shown in FIG. Since these configurations and operations are as described above, detailed description thereof will be omitted.
For each person area of the plurality of images, the feature change model storage unit 102 includes a plurality of axes parallel to a predetermined axis (for example, a horizontal axis and an x axis) defined on the image, and a person area included in each image. , And a feature change amount corresponding to a change in a position on a vertical axis (for example, a vertical axis or a y-axis) of a value of an image feature amount defined on each line segment intersecting with a predetermined axis is stored as a model.
The part separation position extraction processing unit 101 calculates a feature change amount for a human region on an image input from the outside, and calculates the feature change amount and the feature change amount stored in the feature change model storage unit 102. The comparison is performed, the similar change amount is specified, and the peak position where the specified change value is a predetermined value or more is output.
According to the present embodiment, by outputting the peak position of a feature change model in which the change amount of the image feature quantity is similar to the clothes worn by the person on the input image, the parts of the clothes are separated. And can be extracted.
While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-113663 for which it applied on May 20, 2011, and takes in those the indications of all here.

Claims

For each person area on the image, the predetermined value of the feature value calculated for each line segment where a plurality of axes parallel to the predetermined axis on the image and the person area included in the image intersect A feature change model accumulating means for storing a change amount with respect to a change in position on the vertical axis with respect to the axis as a model;
The amount of change is calculated for a human region of an external input image, the amount of change is compared with the amount of change of the model, and a model having a similar amount of change is extracted from the feature change model storage unit And a part separation position extraction processing means for outputting a peak position where the value of the amount of change of the extracted model is a position that is a predetermined value or more
A site separation position extraction device comprising:
The model associates the amount of change with a person attribute, which is information on at least one of clothing type, name, and gender, assigned to the section on the vertical axis divided by the peak position. Including
The part separation position extraction device according to claim 1, wherein the part separation position extraction processing unit further outputs the person attribute of the extracted model.
A person attribute storage means for storing the individual feature amount calculated for each person and the person attribute in association with each other;
Extracting the individual feature amount from the person region of the input image;
A person specifying information extraction collation processing means for acquiring the person attribute corresponding to the individual feature quantity similar to the extracted individual feature quantity from the person attribute storage means;
With
The part separation position extraction processing unit extracts a change amount of the model corresponding to a person attribute similar to the person attribute acquired by the person specifying information extraction collation processing unit from the feature change model storage unit, and extracts the change The part separation position extraction processing device according to claim 2, wherein the comparison is performed between an amount and a change amount calculated from the input image.
For each person area on the image, a feature value calculated on each line segment where a plurality of axes parallel to the vertical axis intersects the person area included in the image is on the predetermined axis. A second feature change model storage unit that stores a change amount (second change amount) with respect to a change in position as a second model;
The part separation / extraction means calculates the second change amount for the person region of the input image, and compares the second change amount with the second change amount of the second model. And extracting a second model having a similar second change amount from the second feature change model accumulating means, and the value of the second change amount of the extracted second model is not less than a predetermined value The part separation position extraction apparatus according to any one of claims 1 to 3, further outputting the second peak position.
The part according to claim 4, wherein the part separation position extraction processing unit divides the person area in the vertical axis direction based on the peak position, and calculates the second change amount for the part of the divided person area. Separation position extraction device.
The model further includes the direction of the person in the person area and the amount of change in association with each other,
The part separation position extraction processing means extracts the orientation of the person from the person area of the input image, compares the orientation of the person with the orientation of the person stored in the feature change model accumulation means, and is similar The part separation position extraction apparatus according to claim 1, wherein the comparison is performed between a change amount of the model corresponding to a direction of a person to be performed and a change amount calculated from the input image.
The part separation position for storing the input image acquisition time when the part separation position extraction apparatus has acquired the input image from the outside and the change amount calculated by the part separation position extraction processing unit with respect to the image in association with each other Storage means;
Person area information storage means for storing the input image acquisition time and the person area of the input image in association with each other;
Further comprising
The part separation position extraction processing means is the input image acquisition time corresponding to a person area similar to the person area of the new input image within a predetermined time when the new input image is input from the outside. Extract a specific time from the person area information storage means,
The change amount corresponding to the input image acquisition time that is within a predetermined time of the specific time is extracted from the part separation position accumulation unit, and the extracted change amount and the change amount calculated by the part separation position extraction processing unit The part separation position extraction apparatus according to any one of claims 1 to 6, wherein the comparison is performed.
8. The site separation position extracting apparatus according to claim 1, wherein the predetermined axis is a vertical axis of a person determined based on the person area.
For each person area on the image, the predetermined value of the feature value calculated for each line segment where a plurality of axes parallel to the predetermined axis on the image and the person area included in the image intersect A computer comprising a feature change model storage means for storing, as a model, a change amount with respect to a change in position on a vertical axis with respect to an axis
The amount of change is calculated for a human region of an input image from the outside, the amount of change is compared with the amount of change stored in the feature change model storage means, and a model having a similar amount of change is obtained. A program storage medium for storing a part separation position extraction program for executing a part separation position extraction processing step for outputting a peak position that is extracted and outputs a peak position where a value of a change amount of the extracted model is a predetermined value or more.
For each person area on the image, the predetermined value of the feature value calculated for each line segment where a plurality of axes parallel to the predetermined axis on the image and the person area included in the image intersect The amount of change with respect to the change in position on the vertical axis relative to the axis is stored as a model,
The amount of change is calculated for a human region of an input image from the outside, the amount of change is compared with the amount of change stored in the feature change model storage means, and a model having a similar amount of change is obtained. A part separation position extraction method for extracting and outputting a peak position where a value of a change amount of the extracted model is a predetermined value or more.