JP7259313B2 - Attribute determination device, attribute determination system, attribute determination method, program and recording medium - Google Patents

Description

The present invention relates to an attribute determination device, an attribute determination system, an attribute determination method, a program, and a recording medium for determining attributes of a person based on an image of the person photographed from above.

2. Description of the Related Art Conventionally, techniques for recognizing attributes such as a person's gender and age based on a person's face image are widely known. However, in a store, for example, in order to recognize the attributes of a person who contacts a certain product group based on the face image, it is necessary to install a camera for acquiring the face image at each sales floor and each display shelf. . In this case, the number of cameras to be installed increases, which not only increases the cost but also increases the psychological burden on the user to whom the camera is directed.

Therefore, from the viewpoint of reducing costs and reducing the psychological burden on the user, a camera is installed on the ceiling of a store, etc., a person is photographed from above in a wide range, and an image including a full-body image of the person is acquired. It is desirable to recognize attributes of a person. In this respect, for example, in the system of Patent Document 1, the entire store is photographed by a main camera installed on the ceiling or on the wall, and the image portion of the person is analyzed, and the customer's face, hairstyle, clothes, height, accessories, shoes, etc. are integrated. I'm trying to guess the gender and age group in consideration of it.

Further, for example, in the technique of Patent Document 2, a model of a person's shape and clothing shapes prepared for each type of clothing are stored in a storage unit, and edges detected in a camera image are compared with the model. A person is detected, and the sex of the person is estimated from the shape of the person. Also, by comparing the clothing shape and the person's shape, the type of clothing worn by the person is estimated, and based on the estimated gender and clothing type, the image is included in the image by gender and clothing type. I am trying to count the number of people.

On the other hand, when a person is photographed in a store and an image is acquired, the background other than the person is often reflected in the image. This background includes people other than the person whose attributes are to be recognized, and products displayed in the sales floor. In this case, the background portion of the image may adversely affect the attribute recognition of the person. For example, there is a high possibility that a man passing in front of a sales floor where women's clothes are displayed will be misidentified as a woman because of the women's clothes reflected in the background.

In this respect, for example, in Patent Document 3, a silhouette of a person is extracted from an input image, and processing such as adding a random number to the feature amount of the portion corresponding to the background is performed to increase the variation of the background and to identify the object. By learning the discrimination parameters, it is possible to obtain high discrimination performance that is less susceptible to the background.

Japanese Patent Application Laid-Open No. 2007-74330 (claims 1 to 3, paragraphs [0021], [0023], see FIG. 1, etc.) Japanese Patent Application Laid-Open No. 2012-173761 (Claim 1, paragraphs [0007] to [0012], see FIGS. 1 to 4, etc.) Japanese Patent Application Laid-Open No. 2008-59110 (Claim 1, 7, 8, paragraphs [0010] to [0012], [0050], see FIGS. 1 to 6, etc.)

However, it is difficult to accurately determine a person's attribute from an image captured by a camera installed on the ceiling or the like. In Patent Literature 1, when it is difficult to identify attributes, a face image captured by an auxiliary camera installed near a display shelf in a sales floor is analyzed in detail to estimate gender and age group. Therefore, the technique of Patent Document 1 still has the problem of imposing a psychological burden on the user who is pointed at the camera (the technique of Patent Document 1 is sufficient as a method of recognizing attributes using only a full-body image of a person as an image). isn't it).

In addition, in a store, the posture of a person whose attributes are to be recognized constantly changes due to his or her movement, taking out products from a product shelf, and the like. In Patent Document 2, clothes are estimated based on whether or not the estimated human shape is matched with a certain clothing shape stored in advance in a storage unit. It becomes difficult, and the clothes may not be recognized. In other words, attributes may not be recognized depending on the posture of the person, and highly versatile attribute recognition cannot be performed.

In addition, when the identification parameter is learned by increasing the variation of the background as in Patent Document 3, a plurality of images are generated in which the silhouette portion of the person is the same and only the feature amount of the background portion is different. If the identification parameters are learned using such images, the identification parameters may overlearn for the same person. Therefore, it becomes difficult to recognize attributes of various persons with high accuracy.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems. To provide an attribute determination device, an attribute determination system, an attribute determination method, a program, and a recording medium capable of highly versatile attribute recognition and highly accurate attribute recognition of various persons. It is in.

An attribute determination device according to one aspect of the present invention includes a position detection unit that detects the position of a control point that defines a skeleton of the person from an image of the person photographed from above, and detection by the position detection unit. and an attribute recognition unit that recognizes the attributes of the person using the result.

An attribute determination system according to another aspect of the present invention includes the attribute determination device described above, and an image acquisition unit that acquires the image by photographing the person from above and outputs data of the image to the attribute determination device. I have.

An attribute determination method according to still another aspect of the present invention includes a position detection step of detecting the position of a control point that defines the skeleton of the person from an image of the person photographed from above; and an attribute recognition step of recognizing the attributes of the person using the detection result of.

A program according to still another aspect of the present invention is a program for causing a computer to execute the above attribute determination method.

A recording medium according to still another aspect of the present invention is a computer-readable recording medium recording the above program.

It is possible to avoid imposing a psychological burden on the user by photographing the face. In addition, it is possible to perform attribute recognition with high versatility that easily copes with changes in a person's posture. Furthermore, attribute recognition can be performed with high accuracy for various persons.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the structure of the outline of the attribute determination system of one embodiment of this invention. It is a flowchart which shows the flow of a process by the attribute recognition method in the said attribute determination system. FIG. 4 is an explanatory diagram showing an example of control points detected in each frame image; FIG. 11 is an explanatory diagram showing an example of control points detected in photographed images of persons having different body types in an attribute determination system according to another embodiment of the present invention; FIG. 11 is a block diagram showing a schematic configuration of an attribute determination system according to still another embodiment of the present invention; 4 is a graph showing a normal distribution curve; It is a flowchart which shows the flow of a process by the attribute recognition method in the said attribute determination system. It is explanatory drawing which shows typically the process by the said attribute recognition method.

Each embodiment of the present invention will be described below with reference to the drawings. In addition, the present invention is not limited to the following contents.

<Embodiment 1>
[Attribute determination system]
FIG. 1 is a block diagram showing a schematic configuration of an attribute determination system 1 of this embodiment. The attribute determination device 1 includes an image acquisition section 2 and an attribute determination device 3 . The image acquisition unit 2 and the attribute determination device 3 are connected via a communication line N so as to be communicable. The communication line N is appropriately selected from, for example, cables, optical fibers, wired LANs (Local Area Networks), wireless LANs, Internet lines, and the like. Details of the image acquisition unit 2 and the attribute determination device 3 will be described below.

(Image acquisition unit)
The image acquiring unit 2 is installed, for example, on the ceiling or wall of the store, and is configured by a camera (image capturing unit) that captures an image of a person in the store from above and acquires temporally different frames of images. By photographing a person from above with the image acquisition unit 2, an image including a full body image of the person is acquired. For example, an RGB camera can be used as the camera that constitutes the image acquisition unit 2 . The RGB camera has pixels corresponding to R (red), G (green), and B (blue), and outputs an electric signal (image data) corresponding to the amount of light received from each pixel. A color image of a person can be obtained by photographing with an RGB camera.

The number of image acquisition units 2 installed in the store is not particularly limited, and may be one or two or more. Image data acquired by at least one image acquisition unit 2 is output to the attribute determination device 3 via a communication line N. FIG.

(attribute determination device)
The attribute determination device 3 is a terminal device that determines the attributes of a person based on the image acquired by the image acquisition unit 2 and input to the attribute determination device 3, that is, the image of each frame obtained by photographing the person from above. Yes, it consists of a personal computer, for example. The attribute determination device 3 may be installed in the same store as the store where the image acquisition unit 2 is installed, or may be installed outside the store so as to be able to communicate with the image acquisition unit 2 .

The attribute determination device 3 has an input unit 11, a storage unit 12, a display unit 13, a communication unit 14, a reading unit 15, a position detection unit 16, an attribute recognition unit 17, and a control unit 18. configured as follows.

The input unit 11 is composed of, for example, a keyboard, a mouse, a touch pad, a touch panel, etc., and receives various instruction inputs from an operator who operates the attribute determination device 3, for example.

The storage unit 12 stores an operation program for operating each unit of the attribute determination device 3, image data input from the image acquisition unit 2, detection results from the position detection unit 16, and attribute recognition by the attribute recognition unit 17. A memory for storing results and the like. The storage unit 12 is configured by, for example, a hard disk, but may be configured by appropriately selecting from recording media such as RAM (Random Access Memory), ROM (Read Only Memory), optical disk, magneto-optical disk, and nonvolatile memory. good.

The display unit 13 is a device that displays various types of information, and is configured by, for example, a liquid crystal display device. The communication unit 14 is an interface including an input/output port for communicating with the outside. The communication unit 14 includes an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like when performing wireless communication with the outside.

The reading unit 15 is a device for reading information recorded on the recording medium R (for example, an operation program of the control unit 18), and is composed of, for example, a disk drive. If the recording medium R is a portable non-volatile memory, the reading unit 15 also includes a connection port into which the connection unit of the non-volatile memory is inserted.

The control section 18 is composed of a central processing unit (CPU) that controls the operation of each section of the attribute determination device 3 , and operates according to an operation program stored in the storage section 12 .

The position detection unit 16 detects the positions of the control points that define the skeleton of the person from the image acquired by the image acquisition unit 2 and input to the attribute determination device 3, that is, the image of the person photographed from above. To detect. The above-mentioned "control points" include human joint points (movable joints connecting bones), the center point of the skull, points at the tips of the bones of the fingers and toes, and the like.

The position detection unit 16 is composed of, for example, the same CPU as the control unit 18 or a separate CPU. Note that a method for detecting the positions of control points (for example, joint points) of a person from an input image has been established with existing software (programs). Therefore, by reading the software with the reading unit 15 and executing the read program with the position detection unit 16, the positions of the control points of the person can be detected from the image of the person photographed from above.

The attribute recognition unit 17 uses the detection result (position information of the control point) from the position detection unit 16 to recognize the attributes of the person. For example, at least one of the person's age, age group, sex, degree of obesity, clothing, attachments, and belongings can be considered as the attributes. Such an attribute recognition unit 17 is composed of, for example, a GPU (Graphics Processing Unit). A GPU is an arithmetic device specialized for real-time image processing.

[Attribute recognition method]
Next, the attribute recognition method of this embodiment will be described. Here, as an example, a case where the attribute recognition unit 17 recognizes gender (male/female) as an attribute of a person will be described. FIG. 2 is a flow chart showing the flow of processing by the attribute recognition method of this embodiment.

First, the image acquisition unit 2 installed in the store captures a person in the store from above to acquire a plurality of frames of images (images including the full body image of the person) (S1; image acquisition step). When the image data of each frame is input to the attribute determination device 3, the position detection unit 16 detects control points that define the skeleton of a person and their positions for each frame image (S2; position detection step). FIG. 3 shows an example of control points detected in the first frame image and control points detected in the second frame image. Black circles in the figure indicate control points.

Subsequently, the attribute recognition unit 17 uses the detection result of the position detection unit 16 to recognize the attributes of the person (S3; attribute recognition step). Specifically, the attribute recognition unit 17 recognizes the attribute based on the time-series variation of the position of the control point.

For example, in FIG. 3, let _P1 be the control point indicating the tip of the finger of the person's right hand, _P2 be the control point (joint point) indicating the elbow of the right arm, and _P3 be the control point indicating the right shoulder. In the image of the first frame, the coordinates of each control point when a predetermined position is taken as the origin are respectively P ₁ =(x ₁₁ , y ₁₁ ), P ₂ =(x ₂₁ , y ₂₁ ), P ₃ = (x ₃₁ , y ₃₁ ). Similarly, in the image of the second frame, the coordinates of each control point with respect to the origin are P ₁ =(x ₁₂ , y ₁₂ ), P ₂ =(x ₂₂ , y ₂₂ ), P ₃ =(x ₃₂ , y ₃₂ ). Let ΔP 1 , _ΔP 2 _, and ΔP ₃ be the amounts of variation of the control points P _{1 to} P _{3 between} the first frame and the second frame, respectively _. ) ² + (y ₁₂ - y ₁₁ ) ² }, ΔP ₂ = √ {(x ₂₂ - x ₂₁ ) ² + (y ₂₂ - y ₂₁ ) ² }, ΔP ₃ = √ {(x ₃₂ - x ₃₁ ) ² +(y ₃₂ −y ₃₁ ) ² }.

For example, men and women walk differently. Men tend to swing their arms more when walking, and the amount of variation ΔP ₁ to ΔP ₃ between frames tends to be greater. On the other hand, in the case of females, there is a tendency that the amount of variation ΔP ₁ to ΔP ₃ between frames is small because the swing of the arms is small during walking. Therefore, the attribute recognizing unit 17 compares the variation amounts ΔP ₁ to ΔP ₃ of the positions of the control points P ₁ to P ₃ with a predetermined threshold to determine whether the person is male or female. (gender) can be recognized.

Note that the stride during walking tends to be large for men and small for women. Therefore, focusing on leg joint points (for example, knees) as control points, the attribute recognizing unit 17 can also recognize gender as an attribute based on time-series fluctuations in the leg joint points. In other words, the attribute recognizing unit 17 can recognize sex as an attribute based on the time-series variation of the joint points that show movements peculiar to males or females during walking.

The recognition result (attribute determination result) by the attribute recognition unit 17 is displayed on the display unit 13, for example (S4; recognition result output step).

As described above, in the present embodiment, the position detection unit 16 detects the positions of the control points that define the skeleton of the person from an image of the person photographed from above (an image including the full-body image of the person). . Then, the attribute recognition unit 17 recognizes the attributes of the person using the detection result of the position detection unit 16, that is, the position information of the control points. By using the position information of the control points, attribute recognition can be performed without using a person's face image at all. As a result, it is possible to avoid imposing a psychological burden on the user (a person in the store) due to photographing the face.

In addition, no matter how the posture of the person changes, that is, no matter how the position of the control point changes due to the change of posture, the position detection unit 16 detects the position of the control point and recognizes the attribute. A unit 17 recognizes the attributes of the person. Therefore, attribute recognition with high versatility that easily copes with changes in the posture of a person is possible.

Furthermore, since the attribute recognition unit 17 recognizes the attributes of the person using the detection result of the position detection unit 16, the attributes of any person can be recognized based on the position information of the control points of the person. can be done. In addition, by recognizing attributes based on the positional information of control points, areas that are not related to control points, that is, areas that are unnecessary for human attribute recognition (for example, background areas) can affect attribute recognition. do not have. Therefore, attribute recognition can be performed with high accuracy for various persons.

In particular, in the present embodiment, the attribute recognition unit 17 can recognize a person's walking style based on time-series changes in the positions of the control points. Therefore, it is possible to recognize the gender (male/female) as a person's attribute based on the recognition result (walking style).

Further, in this embodiment, an RGB camera is used as the image acquisition unit 2 . The attribute determination system 1 of this embodiment can be easily realized by using an RGB camera that is relatively easily available. In Embodiment 3, which will be described later, a feature amount is extracted from an input image by a neural network, and an attribute is recognized based on the extracted feature amount. can do. In this respect, it can be said that the RGB camera is also very suitable for the configuration of the third embodiment.

By the way, as the image acquisition unit 2, a thermal camera or a depth camera may be used. Also, an RGB camera and a thermal camera or a depth camera may be used together. A thermal camera is a camera that detects heat (infrared radiant energy) radiated from an object, converts it into temperature, and displays the temperature distribution as an image. A depth camera is a camera with a built-in depth sensor that acquires depth information. By using a thermal camera or a depth camera, the position detection unit 16 can accurately detect the position of the person's control point from the camera image using temperature distribution or depth information. Therefore, an improvement in the accuracy of attribute recognition by the attribute recognition unit 17 can be expected.

Also, as the image acquisition unit 2, an infrared camera may be used, or a monochrome camera may be used. At this time, an RGB camera and an infrared camera or a monochrome camera may be used together. An infrared camera is a camera that acquires an image by emitting infrared rays (near-infrared rays) and capturing infrared rays reflected by an object. A monochrome camera is a camera that obtains a monochrome (black and white) image by photographing. Infrared cameras and monochrome cameras are very effective in recognizing attributes based on images captured in such dark environments, since they can capture clear images of people even at night.

In this embodiment, an example in which the attribute recognition unit 17 recognizes gender as an attribute of a person has been described. It is also possible to recognize old/middle-aged/young)).

In this embodiment, the attribute determination device 3 acquires image data from the image acquisition unit 2 and performs attribute recognition. may be used for attribute recognition. Further, the attribute determination device 3 may read the data of the image recorded on the recording medium R to recognize the attribute.

<Embodiment 2>
In the present embodiment, in the attribute recognition step of S3 described in Embodiment 1, the attribute recognition unit 17 recognizes the attributes of a person based on the positional relationship of a plurality of control points. Here, as an example, a case where the attribute recognition unit 17 recognizes the degree of obesity as an attribute of a person will be described.

For example, FIG. 4 shows an example of control points detected in an image of a thin person and control points detected in an image of an obese person. Black circles in the figure indicate control points. The position of each control point is the average position over multiple frames when the person is in a stationary state or in a state close to that over multiple frames (the amount of variation in each control point between consecutive frames is less than the threshold). (assuming that the person is not in a walking state).

From the positions of the respective control points shown in FIG. 4, it can be seen that the mutual positional relationships of the shoulders, elbows, fingertips, hip joints, and knees are different between the thin person and the obese person. In particular, it can be seen that the distance between the hip joint and the tip of the finger is significantly different between a lean person and an obese person.

Therefore, the attribute recognition unit 17 recognizes the positional relationship of these control points from the positional coordinates of a plurality of control points (for example, the hip joint and the tip of the finger). For example, the attribute recognition unit 17 obtains the distance between the hip joint and the tip of the finger from the position coordinates of each control point. If the distance is greater than the predetermined range, the attribute recognition unit 17 determines that the person is obese, and if the distance is within the predetermined range, the person is of normal body type. If it is smaller than the range, the person is recognized as thin.

In this way, the attribute recognition unit 17 can recognize a person's body type based on the positional relationship of a plurality of control points. This makes it possible to recognize the degree of obesity (obesity type/standard type/thin type) as an attribute of a person.

Note that the attribute that the attribute recognition unit 17 can recognize based on the positional relationship of the plurality of control points is not limited to the degree of obesity in this embodiment. Based on the same idea as in the present embodiment, the attribute recognition unit 17 may recognize attributes other than the degree of obesity, such as gender (male/female) and age group (adult/child, old/middle-aged/young). It is possible.

<Embodiment 3>
FIG. 5 is a block diagram showing a schematic configuration of the attribute recognition system 1 of this embodiment. The attribute recognition system 1 of this embodiment has the same configuration as that of the first embodiment except that the attribute recognition unit 17 of the attribute determination device 3 is replaced with an attribute recognition unit 17'. Differences from the first embodiment will be described below.

The attribute recognition unit 17 ′ recognizes attributes of a person using the detection result of the position detection unit 16 , similarly to the attribute recognition unit 17 . This attribute recognizing unit 17' is composed of a GPU having functions as a feature quantity extractor 17a, a feature quantity converter 17b, and a discriminator 17c. In particular, the functions of the feature quantity extractor 17a and classifier 17c are realized by a deep neural network (DNN) capable of deep learning.

The feature quantity extractor 17a extracts a feature quantity for recognizing an attribute from an image of a person photographed from above, and generates an n-dimensional (n is a natural number) feature map. The feature quantity converter 17b performs mask processing on the above feature map. More specifically, the feature quantity converter 17b performs the above-described masking process by applying a mask to the feature map in which the degree of decrease in the feature quantity increases with increasing distance from the control point in the image. Here, the above mask has weights that change so that the degree of deterioration of the feature quantity becomes a normal distribution with the control point at the top. The discriminator 17c discriminates attributes based on the masked feature map.

FIG. 6 is a graph showing a normal distribution curve. In the normal distribution curve formula f(x), μ indicates the average and σ indicates the standard deviation. The above mask weight M corresponds to f(x) in FIG. However, 0≤M≤1, and is maximum (M=1) at the position of the control point. Let fn(x, y) be the nth-dimensional feature quantity at the position (x, y) in the image, and let M(x, y) be the weight of the mask to be multiplied by the feature quantity. Then, the feature amount after masking, that is, the feature amount Gn(x, y) for recognizing the attribute is expressed by the following equation.
Gn(x,y)=fn(x,y)×M(x,y)

Next, the attribute recognition method of this embodiment will be described. FIG. 7 is a flow chart showing the flow of processing by the attribute recognition method of this embodiment. FIG. 8 is an explanatory diagram schematically showing processing by the attribute recognition method of this embodiment. It is assumed that the feature quantity extractor 17a and the classifier 17c have been pre-learned by supervised learning (the parameters of each node constituting the neural network are appropriately set (updated)).

First, an image (for example, the first frame) is acquired by photographing a person in the store from above in the image acquisition unit 2 installed in the store (S1; image acquisition step). When the data of the first frame image is input to the attribute determination device 3, the position detection unit 16 detects control points and their positions in the image (S2; position detection step).

Next, the position detection unit 16 recognizes areas such as the head, torso, arms, and legs of the person from the positions of the control points detected in S2, sets a person rectangle including these areas, and sets S1. The image inside the rectangle of the person is extracted from the photographed image acquired in 1 (S2-1; person rectangle image extraction step). Note that the step of S2-1 is not essential and can be omitted.

Subsequently, the attribute recognition unit 17′ recognizes the attribute of the person based on the rectangular image of the person acquired in S2-1 or the photographed image acquired in S1 and the detection result of the position detection unit 16 (S3; attribute recognition process).

Specifically, first, the feature amount extractor 17a extracts feature amounts from an input image (human rectangular image or photographed image), and generates a feature map that is the distribution of the extracted feature amounts (S31; feature amount extraction step ). The feature maps are generated for n dimensions according to the types of feature amounts such as color information and edge information.

Next, the feature quantity converter 17b applies the above-described mask to the n-dimensional feature map to perform mask processing, and reduces the feature quantity at positions farther from the control point (S32; feature quantity conversion process). By this processing, for example, the feature amount of the background portion (including the product on the product shelf) other than the person in the image is reduced. Thereafter, the discriminator 17c discriminates the attributes of the person based on the masked feature map (S33; discrimination step).

The recognition result of the attribute recognition unit 17' (the attribute identification result of the classifier 17c) is displayed, for example, on the display unit 13 (S4; recognition result output step). Then, when the processing is to be continued for the next captured image, the steps after S1 are repeated, and when the next captured image is not to be processed, the series of processing ends (S5).

In an image of a person photographed from above, the more distant the area from the control point, the more likely it is an area unrelated to the person (for example, a background area, which is unlikely to represent an attribute of the person). . The feature quantity converter 17b performs mask processing on the feature map to reduce the feature quantity of regions unrelated to the person away from the control point. Identify attributes based on As a result, it is possible to reduce the influence of the feature amount of the area unrelated to the person on the attribute recognition, and it is possible to accurately identify (recognize) the attribute of the person.

Also, the mask used for the mask processing has weights that change so that the degree of feature amount reduction becomes a normal distribution with the control point at the top. By performing the masking process using the mask by the feature amount converter 17b, the feature amount can be reliably reduced in the area farther from the control point on the feature map. As a result, it is possible to reliably reduce the influence of the feature amount of the area unrelated to the person on attribute recognition, and to reliably improve the accuracy of attribute recognition.

In addition, in this embodiment, the feature amount extractor 17a and the classifier 17c can be learned by using the identification result obtained by reducing the feature amount of the area unrelated to the person as a new correct label. As a result, recognition performance can be improved and versatility can be improved even when recognizing a person's attribute based on an image acquired by photographing in an unknown store.

By the way, the standard deviation σ of the normal distribution, which is the mask weight distribution, may vary according to the positions of the control points in the skeleton of the person. For example, if the control point is the tip of a person's finger bone, the standard deviation σ of the normal distribution with that point as the top is reduced, and if the control point is the center point of the person's skull, You may increase the standard deviation σ of the normal distribution with .

By setting the standard deviation σ of the normal distribution in this way, in the feature map, the range of the region (region related to the person) in which the feature amount is to be retained by the mask processing can be appropriately adjusted according to the position of the human skeleton. can be adjusted. In other words, it is possible to hold the feature amount of the person's head in a wide range and hold the feature amount of the fingers in a narrow range. As a result, the classifier 17c can identify the attribute of the person based only on the feature amount of the area representing the shape of the person, based on the feature map after the mask processing, thereby improving the accuracy of identifying the attribute of the person. can be made

Further, the standard deviation σ of the normal distribution, which is the mask weight distribution, may differ according to the attributes identified by the discriminator 17c. For example, when identifying belongings (for example, pets, handbags, pistols, etc.) as attributes of a person, the standard deviation σ of the normal distribution with the point at the tip of the finger bone of the person as the top may be increased. Also, when gender is identified as a person's attribute, the standard deviation σ of the normal distribution with the point at the tip of the finger bone of the person as the top may be reduced.

By varying the standard deviation .sigma. can be adjusted to As a result, the discriminator 17c can appropriately discriminate the attribute. For example, in the case of identifying whether or not a person possesses a portable item as an attribute, the mask processing holds a feature amount that is wider than the person's fingers. to properly identify the item. On the other hand, when identifying an attribute other than a portable item such as a person's gender as an attribute, since the feature amount of the area of the person's fingers is retained by mask processing, the discriminator 17c uses the portable item based on the feature amount. Attributes other than objects can be appropriately identified.

<Program and recording medium>
The attribute determination device 3 described in each of the above embodiments can be configured by, for example, a computer (PC) in which a predetermined program (application software) is installed. A computer (for example, the control unit 18 as a CPU) reads and executes the above program, so that each unit of the attribute determination device 3 can be operated to execute each process (each process) described above. Such a program is acquired by, for example, downloading from the outside via a network and stored in the storage unit 12 . The program is recorded in a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory), and the computer reads the program from the recording medium and stores the program in the storage unit 12. may

<Supplement>
The attribute determination device, attribute determination system, attribute determination method, program, and recording medium of this embodiment described above can also be expressed as follows.

1. a position detection unit that detects the position of a control point that defines the skeleton of the person from an image of the person photographed from above;
An attribute determination device, comprising: an attribute recognition unit that recognizes the attribute of the person by using the detection result of the position detection unit.

2. 2. The attribute determination device according to 1, wherein the attribute recognition unit recognizes the attribute based on a time-series change in the position of the control point.

3. 2. The attribute determination device according to 1, wherein the attribute recognition unit recognizes the attribute based on the positional relationship of the plurality of control points.

4. The attribute recognition unit
a feature quantity extractor that extracts a feature quantity for recognizing the attribute from the image and generates a feature map;
a feature quantity converter that performs mask processing on the feature map;
a classifier that identifies the attribute based on the masked feature map;
2. The attribute determination according to 1, wherein the feature value converter performs the mask processing by applying a mask to the feature map in which the degree of decrease in the feature value increases with the position away from the control point. Device.

5. 5. The attribute determination device according to 4 above, wherein the mask has weights that vary so that the degree of deterioration of the feature quantity has a normal distribution with the control point at the top.

6. 6. The attribute determination device according to 5, wherein the standard deviation of the normal distribution differs according to the position of the control point in the skeleton of the person.

7. 7. The attribute determination device according to 5 or 6, wherein the standard deviation of the normal distribution differs according to the attribute identified by the discriminator.

8. 8. The attribute determination device according to any one of 1 to 7, wherein the attribute of the person is at least one of the person's age, age group, sex, degree of obesity, clothing, attachment, and belongings. .

9. 9. The attribute determination device according to any one of 1 to 8;
An attribute determination system, comprising: an image acquisition unit that captures an image of a person from above to acquire the image, and outputs data of the image to the attribute determination device.

10. 10. The attribute determination system as described in 9 above, wherein the image acquisition unit includes an RGB camera.

11. 11. The attribute determination system according to 9 or 10, wherein the image acquisition unit includes a thermal camera or a depth camera.

12. 12. The attribute determination system according to any one of 9 to 11, wherein the image acquisition unit includes an infrared camera or a monochrome camera.

13. a position detection step of detecting the positions of control points that define the skeleton of the person from an image of the person photographed from above;
and an attribute recognition step of recognizing the attributes of the person using the detection result of the position detection unit.

14. 14. The attribute determination method according to 13 above, wherein, in the attribute recognition step, the attribute is recognized based on a time-series change in the position of the control point.

15. 14. The attribute determination method according to 13 above, wherein in the attribute recognition step, the attribute is recognized based on a positional relationship between a plurality of the control points.

16. The attribute recognition step includes
a feature quantity extraction step of extracting a feature quantity for recognizing the attribute from the image and generating a feature map;
a feature amount conversion step of performing mask processing on the feature map;
an identification step of identifying the attribute based on the masked feature map;
14. The attribute determination according to 13 above, wherein in the feature value conversion step, the mask processing is performed by applying a mask to the feature map in which the degree of decrease in the feature value increases with the position away from the control point. Method.

17. 17. A program for causing a computer to execute the attribute determination method according to any one of 13 to 16 above.

18. 18. A computer-readable recording medium recording the program according to 17 above.

Although the embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and can be implemented by being expanded or modified without departing from the gist of the invention.

INDUSTRIAL APPLICABILITY The present invention can be used for a device that determines attributes of a person based on an image of the person photographed from above.

1 attribute determination system 2 image acquisition unit 3 attribute determination device 16 position detection unit 17 attribute recognition unit 17' attribute recognition unit 17a feature amount extractor 17b feature amount converter 17c classifier

Claims (14)

a position detection unit that detects the position of a control point that defines the skeleton of the person from an image of the person photographed from above;
an attribute recognition unit that recognizes the attributes of the person photographed from above using the detection result of the position detection unit;
The attributes of the person include at least one of the age group and gender of the person ,
The attribute recognizing unit recognizes the attribute based on the time-series variation of the position of the control point, and determines an amount of variation between temporally different frames and a predetermined threshold value of the position of the control point. An attribute determination device that recognizes the attribute by comparison . a position detection unit that detects the position of a control point that defines the skeleton of the person from an image of the person photographed from above;
an attribute recognition unit that recognizes attributes of the person using the detection result of the position detection unit;
The attribute recognition unit
a feature quantity extractor that extracts a feature quantity for recognizing the attribute from the image and generates a feature map;
a feature quantity converter that performs mask processing on the feature map;
a classifier that identifies the attribute based on the masked feature map;
The attribute determination device, wherein the feature value converter performs the mask processing by applying a mask to the feature map in which the degree of decrease in the feature value increases with the position away from the control point. 3. The attribute determination apparatus according to claim 2, wherein the mask has weights that vary so that the degree of deterioration of the feature quantity has a normal distribution with the control point at the top. 4. The attribute determination device according to claim 3, wherein the standard deviation of said normal distribution differs according to the position of said control point in said person's skeleton. 5. The attribute determination device according to claim 3, wherein the standard deviation of said normal distribution differs according to said attribute identified by said discriminator. 6. The attribute determination according to any one of claims 2 to 5, wherein the attributes of the person are at least one of the person's age, age group, sex, degree of obesity, clothes, attachments, and belongings. Device. an attribute determination device according to any one of claims 1 to 6;
An attribute determination system, comprising: an image acquisition unit that captures an image of a person from above to acquire the image, and outputs data of the image to the attribute determination device. 8. The attribute determination system according to claim 7, wherein said image acquisition unit includes an RGB camera. 9. The attribute determination system according to claim 7, wherein the image acquisition unit includes a thermal camera or a depth camera. 10. The attribute determination system according to any one of claims 7 to 9, wherein the image acquisition unit includes an infrared camera or a monochrome camera. a position detection step of detecting the positions of control points that define the skeleton of the person from an image of the person photographed from above;
an attribute recognition step of recognizing the attributes of the person photographed from above using the detection result of the position detection step;
The attributes of the person include at least one of the age group and gender of the person,
In the attribute recognition step, the attribute is recognized based on the time-series variation of the position of the control point, and the amount of variation of the position of the control point between temporally different frames and a predetermined threshold are determined. An attribute determination method, wherein the attribute is recognized by comparison. a position detection step of detecting the positions of control points that define the skeleton of the person from an image of the person photographed from above;
an attribute recognition step of recognizing attributes of the person using the detection result of the position detection step;
The attribute recognition step includes
a feature quantity extraction step of extracting a feature quantity for recognizing the attribute from the image and generating a feature map;
a feature amount conversion step of performing mask processing on the feature map;
an identification step of identifying the attribute based on the masked feature map;
In the feature value conversion step, the attribute determination method is characterized in that the mask processing is performed by applying a mask to the feature map in which the degree of decrease in the feature value increases with the position away from the control point. A program for causing a computer to execute the attribute determination method according to claim 11 or 12. A computer-readable recording medium on which the program according to claim 13 is recorded.

Images