JP2020098474A - Attribute determination device, attribute determination system, attribute determination method, program, and recording medium - Google Patents

Abstract

To avoid putting any mental burden on a user through photographing of a face, to enable attribute recognition with a high versatility so as to easily correspond to a posture change of a person, and to enable attribute recognition of various people with high precision.SOLUTION: An attribute determination device 3 includes a position detection unit 16 and an attribute recognition unit 17. From an image obtained by photographing a person from above, the position detection unit 16 detects the position of a control point that defines the skeleton of the person. By using the detection result obtained by the position detection unit 16, the attribute recognition unit 17 recognizes the attribute of the person.SELECTED DRAWING: Figure 1

Description

The present invention relates to an attribute determination device, an attribute determination system, an attribute determination method, a program, and a recording medium that determine the attributes of a person based on an image of a person captured from above.

Conventionally, a technique of recognizing attributes such as gender and age of a person based on a face image of the person is widely known. However, for example, in a store, in order to recognize an attribute of a person who contacts a certain product group based on a face image, it is necessary to install a camera for acquiring a face image in each sales floor or each display shelf. .. In this case, the number of cameras installed increases, which not only increases the cost but also increases the psychological burden on the user who points the camera.

Therefore, from the viewpoint of cost reduction and user's psychological burden reduction, a camera is installed on the ceiling of a store, a person is photographed over a wide area from above, and an image including a full-body image of the person is acquired. It is desirable to recognize the attributes of people. In this respect, for example, in the system of Patent Document 1, the main camera installed on the ceiling or the wall surface is used to photograph the entire area of the store to analyze the image portion of the person, and the face, hairstyle, clothes, height, accessories, shoes, etc. of the customer are integrated. The gender and age group are presumed in consideration.

Further, for example, in the technique of Patent Document 2, a human-shaped template and a clothing shape prepared for each type of clothing are stored in a storage unit, and edges detected in a camera image are compared with the template. A person is detected and the gender of the person is estimated from the shape of the person. In addition, by comparing the clothing shape and the person shape, the type of clothing worn by the person is estimated, and based on the estimated gender and clothing type, it is included in the image by gender and clothing type. I try to count the number of people.

On the other hand, when a person is photographed and an image is acquired in a store, a background other than the person is often reflected in the image. This background includes persons other than the person whose attribute is to be recognized, and products displayed in the sales floor. In this case, it is conceivable that the background portion of the image adversely affects the attribute recognition itself of the person. For example, there is a high possibility that a man passing in front of a sales floor where clothes for women are displayed will be mistakenly recognized as a woman because of the clothes for women reflected in the background.

In this regard, for example, in Patent Document 3, for extracting a silhouette of a person from an input image and performing a process such as adding a random number to a feature amount of a portion corresponding to the background, the variation of the background is increased and an object is identified. By learning the discrimination parameter, high discrimination performance that is not easily influenced by the background is obtained.

JP-A-2007-74330 (see claims 1 to 3, paragraphs [0021] and [0023], FIG. 1 and the like) JP 2012-173761 A (see claim 1, paragraphs [0007] to [0012], FIGS. 1 to 4, etc.) Japanese Unexamined Patent Application Publication No. 2008-59110 (Claims 1, 7, 8 and paragraphs [0010] to [0012], [0050], FIGS. 1 to 6 and the like)

However, the method of accurately determining the attribute of a person from an image captured by a camera installed on a ceiling or the like is difficult. In Patent Document 1, when it is difficult to specify the attribute, a facial image taken by an auxiliary camera installed near the display shelf in the sales floor is analyzed in detail to estimate the sex and age group. Therefore, the technique of Patent Document 1 still has a problem of giving a psychological burden to the user who points the camera (the technique of Patent Document 1 is sufficient as a method of recognizing attributes by using only a full-body image of a person as an image). is not).

Further, in the store, the posture of the person whose attributes are to be recognized constantly changes due to his/her movement, taking out the product from the product shelf, or the like. In Patent Document 2, since the clothing is estimated based on whether or not the estimated person shape matches a certain clothing shape stored in the storage unit in advance, when the posture of the person changes, the person shape and the clothing shape are matched. Sometimes it becomes difficult and the clothes cannot be recognized. In other words, there are cases where the attributes cannot be recognized depending on the posture of the person, and it is not possible to perform highly versatile attribute recognition.

Further, in the case of learning the identification parameter by increasing the variation of the background as in Patent Document 3, a plurality of images in which the silhouette portion of the person is the same and only the characteristic amount of the background portion is different are generated. If the identification parameter is learned using such an image, there is a possibility that a person having the same identification parameter may overlearn. Therefore, it is difficult to recognize the attributes of various persons with high accuracy.

The present invention has been made to solve the above problems, and an object thereof is to avoid giving a psychological burden to a user by photographing a face, and easily cope with a change in a posture of a person. To provide an attribute determination device, an attribute determination system, an attribute determination method, a program, and a recording medium which can perform attribute recognition with high versatility and can perform attribute recognition with high accuracy for various persons. It is in.

An attribute determination device according to one aspect of the present invention includes a position detection unit that detects a position of a control point that is a point that defines the skeleton of the person from an image obtained by photographing the person from above, and detection by the position detection unit. An attribute recognition unit that recognizes the attribute of the person using the result is provided.

An attribute determination system according to another aspect of the present invention includes the attribute determination device described above, and an image acquisition unit that acquires the image by photographing a person from above and outputs the image data to the attribute determination device. I have it.

An attribute determining method according to still another aspect of the present invention is a position detecting step of detecting a position of a control point which is a point defining the skeleton of the person from an image obtained by photographing the person from above, and the position detecting section. The attribute recognition step of recognizing the attribute of the person using the detection result of 1.

A program according to still another aspect of the present invention is a program for causing a computer to execute the above-described attribute determination method.

A recording medium according to still another aspect of the present invention is a computer-readable recording medium in which the above program is recorded.

It is possible to avoid giving a psychological burden to the user by photographing the face. In addition, it is possible to perform highly versatile attribute recognition that easily responds to changes in the posture of a person. Furthermore, attribute recognition can be performed with high accuracy for various persons.

It is a block diagram which shows the schematic structure of the attribute determination system of one embodiment of this invention. It is a flowchart which shows the flow of a process by the attribute recognition method in the said attribute determination system. It is explanatory drawing which shows the example of the control point detected in the image of each frame. It is explanatory drawing which shows the example of the control point detected in the picked-up image of the person with a different figure in the attribute determination system of other embodiment of this invention. It is a block diagram which shows the schematic structure of the attribute determination system of other embodiment of this invention. It is a graph which shows a normal distribution curve. It is a flowchart which shows the flow of a process by the attribute recognition method in the said attribute determination system. It is explanatory drawing which shows the process by the said attribute recognition method typically.

Each embodiment of the present invention will be described below with reference to the drawings. The present invention is not limited to the following contents.

<Embodiment 1>
[Attribute determination system]
FIG. 1 is a block diagram showing a schematic configuration of the attribute determination system 1 of this embodiment. The attribute determination device 1 is configured to include an image acquisition unit 2 and an attribute determination device 3. The image acquisition unit 2 and the attribute determination device 3 are communicably connected via a communication line N. The communication line N is configured by appropriately selecting, for example, a cable, an optical fiber, a wired LAN (Local Area Network), a wireless LAN, an Internet line, or the like. Details of the image acquisition unit 2 and the attribute determination device 3 will be described below.

(Image acquisition unit)
The image acquisition unit 2 is installed on, for example, a ceiling or a wall of a store, and is configured by a camera (imaging unit) that captures an image of a person in the store from above and acquires images of temporally different frames. By capturing the person from above by the image acquisition unit 2, an image including the whole-body image of the person is acquired. As a camera that constitutes the image acquisition unit 2, for example, an RGB camera can be used. The RGB camera has pixels corresponding to R (red), G (green), and B (blue), and each pixel outputs an electric signal (image data) according to the amount of received light. A color image of a person can be obtained by photographing with an RGB camera.

The number of image acquisition units 2 installed in the store is not particularly limited, and may be one or two or more. The image data acquired by at least one image acquisition unit 2 is output to the attribute determination device 3 via the communication line N.

(Attribute determination device)
The attribute determining device 3 is a terminal device that determines an attribute of a person based on an image acquired by the image acquiring unit 2 and input to the attribute determining device 3, that is, an image of each frame in which a person is photographed from above. Yes, for example, a personal computer. The attribute determination device 3 may be installed in the same store as the store in which the image acquisition unit 2 is installed, or may be installed outside the store so as to be communicable with the image acquisition unit 2.

The attribute determination device 3 includes an input unit 11, a storage unit 12, a display unit 13, a communication unit 14, a reading unit 15, a position detection unit 16, an attribute recognition unit 17, and a control unit 18. Is configured.

The input unit 11 includes, for example, a keyboard, a mouse, a touch pad, a touch panel, and the like, and receives various instruction inputs by an operator who operates the attribute determination device 3, for example.

The storage unit 12 recognizes an attribute of the operation program for operating each unit of the attribute determination device 3, the image data input from the image acquisition unit 2, the detection result of the position detector 16, and the attribute recognition unit 17. This is a memory for storing results and the like. The storage unit 12 is formed of, for example, a hard disk, but may be formed by appropriately selecting from a recording medium such as a RAM (Random Access Memory), a ROM (Read Only Memory), an optical disk, a magneto-optical disk, and a nonvolatile memory. Good.

The display unit 13 is a device that displays various types of information, and is configured by, for example, a liquid crystal display device. The communication unit 14 is an interface including an input/output port for communicating with the outside. The communication unit 14 is configured to include an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like when performing wireless communication with the outside.

The reading unit 15 is a device that reads information (for example, an operation program of the control unit 18) recorded on the recording medium R, and is configured by a disk drive, for example. When the recording medium R is a portable non-volatile memory, the reading unit 15 also includes a connection port into which the non-volatile memory connection unit is pointed.

The control unit 18 is configured by a central processing unit (CPU) that controls the operation of each unit of the attribute determination device 3, and operates according to the operation program stored in the storage unit 12.

The position detection unit 16 obtains the position of the control point, which is a point defining the skeleton of the person, from the image acquired by the image acquisition unit 2 and input to the attribute determination device 3, that is, the image obtained by photographing the person from above. To detect. Note that the above-mentioned “control point” includes a joint point of a person (a movable joint that connects bones to each other), a central point of a skull, a point of a tip of a bone of a toe, and the like.

The position detector 16 is composed of, for example, a CPU that is the same as or separate from the controller 18. The method of detecting the position of the control point (for example, joint point) of the person from the input image is established by existing software (program). Therefore, by reading the above software by the reading unit 15 and executing the read program by the position detecting unit 16, the position of the control point of the person can be detected from the image obtained by photographing the person from above.

The attribute recognition unit 17 recognizes the attribute of a person by using the detection result (position information of control points) by the position detection unit 16. As the above attribute, for example, at least one of the age, age group, sex, obesity, clothing, worn article, and portable article of a person can be considered. The attribute recognition unit 17 as described above is configured by, for example, a GPU (Graphics Processing Unit). The GPU is a computing device specialized for real-time image processing.

[Attribute recognition method]
Next, the attribute recognition method of this embodiment will be described. Here, as an example, a case where the attribute recognition unit 17 recognizes gender (male/female) as the attribute of a person will be described. FIG. 2 is a flowchart showing the flow of processing by the attribute recognition method of this embodiment.

First, the image acquisition unit 2 installed in a store captures an image of a person in the store from above to obtain images of a plurality of frames (images including the whole-body image of the person) (S1; image acquisition step). When the data of the image of each frame is input to the attribute determination device 3, the position detection unit 16 detects a control point that is a point defining the skeleton of the person and its position in the image of each frame (S2; position). Detection step). FIG. 3 shows an example of control points detected in the image of the first frame and control points detected in the image of the second frame. The black circles in the figure indicate control points.

Then, the attribute recognition unit 17 recognizes the attribute of the person using the detection result of the position detection unit 16 (S3; attribute recognition step). Specifically, the attribute recognition unit 17 recognizes the attribute based on the time-series fluctuation of the position of the control point.

For example, in FIG. 3, the control point indicating the tip of the finger of the person's right hand is P ₁ , the control point indicating the elbow of the right arm (joint point) is P _2, and the control point indicating the right shoulder is P ₃ . In the image of the first frame, the coordinates of each control point when a predetermined position is taken as the origin are P ₁ =(x ₁₁ , y ₁₁ ), P ₂ =(x ₂₁ , y ₂₁ ), P _{3 respectively.} =(x ₃₁ , y ₃₁ ). Similarly, in the image of the second frame, the coordinates of each control point with respect to the origin are respectively P ₁ =(x ₁₂ , y ₁₂ ), P ₂ =(x ₂₂ , y ₂₂ ), P ₃ =(x ₃₂ , Y ₃₂ ). Letting ΔP ₁ , ΔP ₂ , and ΔP ₃ be the fluctuation amounts of the control points P _{1 to} P ₃ between the first frame and the second frame, respectively, ΔP ₁ =√{(x ₁₂ −x _{11 ^{) 2 + (y 12 -y 11}} ) 2}, ΔP 2 = √ {(x 22 -x 21) 2 + (y 22 -y 21) 2}, ΔP 3 = √ {(x 32 -x 31) 2 +(y ₃₂ −y ₃₁ ) ² }.

For example, a man and a woman have different walking styles. In the case of a man, the arm swings greatly during walking, and the fluctuation amounts ΔP _{1 to} ΔP ₃ between frames tend to be large. On the other hand, in the case of a woman, the swing of the arm during walking is small, and the variation amounts ΔP _{1 to} ΔP ₃ between frames tend to be small. Therefore, the attribute recognizing unit 17, by comparing the variation ΔP ₁ ~ΔP ₃ positions of the control points P ₁ to P ₃ with a predetermined threshold value, or the person is a woman or a man, that is, the attribute Can recognize (gender).

The stride length during walking tends to be large for men and small for women. Therefore, when focusing on the joint point of the leg (for example, the knee) as the control point, the attribute recognition unit 17 can also recognize the gender as the attribute based on the time-series variation of the joint point of the leg. That is, the attribute recognition unit 17 can recognize the gender as the attribute based on the time-series variation of the joint points that show the movement peculiar to the male or the female during walking.

The recognition result (attribute determination result) by the attribute recognition unit 17 is displayed, for example, on the display unit 13 (S4; recognition result output step).

As described above, in the present embodiment, the position detection unit 16 detects the position of the control point, which is a point defining the skeleton of the person, from the image of the person photographed from above (the image including the whole-body image of the person). .. Then, the attribute recognition unit 17 recognizes the attribute of the person using the detection result of the position detection unit 16, that is, the position information of the control point. By using the position information of the control points, attribute recognition can be performed without using the face image of the person. As a result, it is possible to avoid giving a psychological burden to the user (person in the store) by photographing the face.

In addition, no matter how the posture of the person changes, that is, how the position of the control point changes due to the change of the posture, the position detection unit 16 detects the position of the control point and performs attribute recognition. The unit 17 recognizes the attribute of the person. Therefore, it is possible to easily recognize a general-purpose attribute, which corresponds to a change in the posture of a person.

Further, since the attribute recognition unit 17 recognizes the attribute of the person using the detection result of the position detection unit 16, any attribute can be recognized based on the position information of the control point of the person. You can Further, by recognizing the attributes based on the position information of the control points, an area unrelated to the control points, that is, an area not necessary for the attribute recognition of a person (for example, a background area) may affect the attribute recognition. Absent. Therefore, attribute recognition can be performed with high accuracy for various persons.

In particular, in the present embodiment, the attribute recognition unit 17 can recognize the way a person walks based on the time-series variation of the position of the control point. Therefore, the gender (male/female) as the attribute of the person can be recognized based on the recognition result (walking style).

Further, in this embodiment, an RGB camera is used as the image acquisition unit 2. The attribute determination system 1 of the present embodiment can be easily realized by using an RGB camera that is relatively easily available. In the third embodiment described later, the feature amount is extracted from the input image by the neural network and the attribute is recognized based on the extracted feature amount. However, if the color image is used as the input image, the feature amount is accurately extracted. can do. From this point, it can be said that the RGB camera is very suitable for the configuration of the third embodiment.

By the way, as the image acquisition unit 2, a thermal camera or a depth camera may be used. Further, the RGB camera and the thermal camera or the depth camera may be used together. A thermal camera is a camera that detects heat radiated from an object (infrared radiation energy), converts it into temperature, and displays the temperature distribution as an image. The depth camera is a camera with a built-in depth sensor that acquires depth information. By using the thermal camera or the depth camera, the position detection unit 16 can accurately detect the position of the control point of the person from the camera image using the temperature distribution or the depth information. Therefore, it is expected that the attribute recognition accuracy of the attribute recognition unit 17 will be improved.

Further, as the image acquisition unit 2, an infrared camera or a monochrome camera may be used. At this time, an RGB camera and an infrared camera or a monochrome camera may be used together. An infrared camera is a camera that emits infrared rays (near infrared rays) and captures an image by capturing reflection of infrared rays on an object. The monochrome camera is a camera that acquires a monochrome (black and white) image by shooting. The infrared camera and the monochrome camera can photograph a person even at night and acquire a clear image, and therefore are very effective for attribute recognition based on the image acquired in such a dark environment.

In the present embodiment, an example has been described in which the attribute recognition unit 17 recognizes gender as an attribute of a person, but based on the same idea as in the present embodiment, attributes other than gender (for example, age group (adult/child, It is also possible to recognize old/middle-aged/young)).

In the present embodiment, an example has been described in which the attribute determination device 3 acquires image data from the image acquisition unit 2 and performs attribute recognition. However, for example, image data transmitted from another terminal device via a communication line. You may perform attribute recognition using. Further, the attribute determination device 3 may read the data of the image recorded on the recording medium R to perform the attribute recognition.

<Second Embodiment>
In the present embodiment, in the attribute recognition step of S3 described in the first embodiment, the attribute recognition unit 17 recognizes a person's attribute based on the positional relationship between a plurality of control points. Here, as an example, a case will be described in which the attribute recognition unit 17 recognizes the degree of obesity as a person attribute.

For example, FIG. 4 shows an example of control points detected in an image of a thin person and a control point detected in an image of an obese person. The black circles in the figure indicate control points. The position of each control point is the average position among a plurality of frames when the person is in a stationary state or a state close to that over a plurality of frames (the amount of variation between consecutive frames of each control point is less than or equal to a threshold). Yes (assuming that the person is not walking).

From the positions of the control points shown in FIG. 4, it is understood that the lean type person and the obese type person have different positional relationships among the shoulders, elbows, finger tips, hip joints, and knees. In particular, it can be seen that the distance between the hip joint and the tips of the fingers differs greatly between a lean person and an obese person.

Therefore, the attribute recognition unit 17 recognizes the positional relationship between these control points from the position coordinates of the plurality of control points (for example, the hip joint and the tips of the fingers). For example, the attribute recognition unit 17 obtains the distance between the hip joint and the tip of the finger from the position coordinates of each control point. When the distance is larger than the predetermined range, the attribute recognition unit 17 determines that the person is obese, and when the distance is within the predetermined range, the person is standard and the distance is predetermined. If it is smaller than the range, the person is recognized to be thin.

In this way, the attribute recognition unit 17 can recognize the body type of the person based on the positional relationship between the plurality of control points. This makes it possible to recognize the degree of obesity (obesity type/standard body type/skinny type) as an attribute of a person.

The attribute that the attribute recognition unit 17 can recognize based on the positional relationship between the plurality of control points is not limited to the obesity degree of the present embodiment. Based on the same idea as this embodiment, the attribute recognition unit 17 may recognize attributes other than the degree of obesity, such as gender (male/female), age group (adult/child, old/middle-aged/young). It is possible.

<Third Embodiment>
FIG. 5 is a block diagram showing a schematic configuration of the attribute recognition system 1 of this embodiment. The attribute recognition system 1 of the present embodiment has the same configuration as that of the first embodiment except that the attribute recognition unit 17 of the attribute determination device 3 is replaced with an attribute recognition unit 17′. The points different from the first embodiment will be described below.

The attribute recognition unit 17 ′, like the attribute recognition unit 17, recognizes the attribute of a person using the detection result of the position detection unit 16. The attribute recognition unit 17' is composed of a GPU having the functions of a feature quantity extractor 17a, a feature quantity converter 17b, and a discriminator 17c. Particularly, the functions of the feature quantity extractor 17a and the classifier 17c are realized by a deep neural network (DNN) capable of deep learning.

The feature amount extractor 17a extracts a feature amount for recognizing an attribute from an image of a person photographed from above and generates an n-dimensional (n is a natural number) feature map. The feature amount converter 17b performs a mask process on the feature map. More specifically, the feature quantity converter 17b performs the mask process by applying a mask to the feature map in which the degree of decrease in the feature quantity increases with the distance from the control point in the image. Here, the mask has a weight that changes so that the degree of decrease in the feature amount has a normal distribution with the control point as the apex. The discriminator 17c discriminates an attribute based on the masked feature map.

FIG. 6 is a graph showing a normal distribution curve. In the equation f(x) of the normal distribution curve, μ represents the average and σ represents the standard deviation. The weight M of the above mask corresponds to f(x) in FIG. However, 0≦M≦1, which is the maximum (M=1) at the position of the control point. When the feature quantity of the nth dimension at the position (x, y) in the image is fn(x, y) and the weight of the mask for multiplying the feature quantity is M(x, y), the feature quantity converter 17b When the masking process is performed by, the feature amount after the masking process, that is, the feature amount Gn(x, y) for recognizing the attribute is expressed by the following formula.
Gn(x,y)=fn(x,y)×M(x,y)

Next, the attribute recognition method of this embodiment will be described. FIG. 7 is a flowchart showing the flow of processing by the attribute recognition method of this embodiment. In addition, FIG. 8 is an explanatory diagram schematically showing processing by the attribute recognition method of the present embodiment. It is assumed that the feature quantity extractor 17a and the discriminator 17c have been preliminarily learned by supervised learning (parameters of each node forming the neural network are appropriately set (updated)).

First, the image acquisition unit 2 installed in a store captures an image (for example, the first frame) of a person in the store from above (S1; image acquisition step). When the data of the image of the first frame is input to the attribute determination device 3, the position detection unit 16 detects the control point and the position of the image (S2; position detection step).

Next, the position detection unit 16 recognizes areas such as a person's head, body, arms, and legs from the positions of the control points detected in S2, sets a person rectangle including these areas, and sets S1. An image inside a person rectangle is extracted from the captured image acquired in (S2-1; person rectangle image extraction step). The step S2-1 is not essential and can be omitted.

Subsequently, the attribute recognition unit 17′ recognizes the attribute of the person based on the person rectangular image acquired in S2-1 or the captured image acquired in S1 and the detection result of the position detection unit 16 (S3; Attribute recognition process).

Specifically, first, the feature quantity extractor 17a extracts the feature quantity from the input image (human rectangular image or photographed image) and generates a feature map which is a distribution of the extracted feature quantity (S31; feature quantity extracting step). ). The feature map is generated for n dimensions according to the types of feature amounts such as color information and edge information.

Next, the feature quantity converter 17b performs mask processing by applying the above-described mask to the above n-dimensional feature map, and reduces the feature quantity at positions farther from the control point (S32; feature quantity conversion). Process). By this processing, for example, the feature amount of the background portion (including the product on the product shelf) other than the person in the image is reduced. Then, the discriminator 17c discriminates the attributes of the person based on the masked feature map (S33; discriminating step).

The recognition result by the attribute recognition unit 17' (the attribute recognition result by the classifier 17c) is displayed, for example, on the display unit 13 (S4; recognition result output step). Then, when the process is continued for the next captured image, the steps from S1 are repeated, and when the process is not performed for the next captured image, the series of processes is finished (S5).

In an image of a person photographed from above, a region farther from the control point is more likely to be a region not related to the person (for example, a region such as a background region, which is unlikely to represent a person's attribute). .. The feature amount converter 17b performs a mask process on the feature map to reduce the feature amount of a region distant from the control point and not related to a person, and the discriminator 17c converts the feature map after the mask process to the feature map. Identify attributes based on. As a result, it is possible to reduce the influence of the feature amount of the area unrelated to the person on the attribute recognition, and it is possible to accurately identify (recognize) the person's attribute.

Further, the mask used for the mask processing has a weight that changes so that the degree of decrease in the feature amount becomes a normal distribution with the control point as the apex. Since the feature amount converter 17b performs the mask process using the mask, the feature amount can be surely reduced in the region farther from the control point in the feature map. As a result, it is possible to reliably reduce the influence of the feature amount of the area unrelated to the person on the attribute recognition, and to reliably improve the accuracy of the attribute recognition.

Further, in the present embodiment, the feature amount extractor 17a and the discriminator 17c can be trained by newly using the identification result obtained by reducing the feature amount of the area unrelated to the person as the correct answer label. As a result, even when recognizing a person's attribute based on an image acquired by shooting in an unknown store, recognition performance can be improved and generalization can be improved.

By the way, the standard deviation σ of the normal distribution, which is the distribution of the weights of the mask, may be different depending on the position of the control point in the skeleton of the person. For example, if the control point is the point of the tip of the bone of a person's finger, the standard deviation σ of the normal distribution with that point as the apex is reduced, and if the control point is the center point of the person's skull, its center point The standard deviation σ of the normal distribution with the peak at may be increased.

By setting the standard deviation σ of the normal distribution in this way, in the feature map, the range of the region (region related to the person) in which the feature amount is retained by the mask processing is appropriately set according to the position of the human skeleton. Can be adjusted. That is, the feature amount can be held in a wide range for the head of the person, and the feature amount can be held in a narrow range for the fingers. As a result, the classifier 17c can identify the attribute of the person based only on the feature amount of the area representing the shape of the person based on the feature map after the mask processing, and improve the identification accuracy of the attribute of the person. Can be made.

Further, the standard deviation σ of the normal distribution, which is the distribution of the weight of the mask, may be different according to the attribute identified by the discriminator 17c. For example, when a portable object (for example, a pet, a bag, a pistol, etc.) is identified as the attribute of a person, the standard deviation σ of the normal distribution having the point at the tip of the bone of the person's finger as the apex may be increased. When gender is identified as a person's attribute, the standard deviation σ of the normal distribution having the point at the tip of the bone of the person's finger as the apex may be reduced.

By varying the standard deviation σ of the normal distribution according to the attribute to be identified in this manner, the feature quantity converter 17b appropriately sets the range in which the feature quantity is held by the mask processing on the feature map according to the attribute to be identified. Can be adjusted to. As a result, the discriminator 17c can appropriately discriminate the above attributes. For example, when identifying whether or not a person carries a portable object as an attribute, since the masking process holds a feature amount in a range wider than that of a person's finger, the discriminator 17c uses the feature amount based on the above feature amount. Therefore, it is possible to properly identify the portable object. On the other hand, when identifying an attribute other than a portable object such as the gender of a person as the attribute, the masking process holds the feature amount of the area of the finger of the person. Attributes other than things can be properly identified.

<Program and recording medium>
The attribute determination device 3 described in each of the above embodiments can be configured by, for example, a computer (PC) in which a predetermined program (application software) is installed. When the computer (for example, the control unit 18 as a CPU) reads and executes the program, each unit of the attribute determination device 3 can be operated to execute each process (each process) described above. Such a program is acquired by being downloaded from the outside via a network, for example, and is stored in the storage unit 12. The program is recorded in a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory), and the computer reads the program from the recording medium and stores it in the storage unit 12. May be.

<Supplement>
The attribute determination device, the attribute determination system, the attribute determination method, the program, and the recording medium of the present embodiment described above can also be expressed as follows.

1. A position detection unit that detects the position of a control point, which is a point that defines the skeleton of the person, from an image obtained by photographing the person from above,
An attribute determination device, comprising: an attribute recognition unit that recognizes an attribute of the person by using a detection result of the position detection unit.

2. 2. The attribute determination device according to 1, wherein the attribute recognition unit recognizes the attribute based on a time-series variation of the position of the control point.

3. The attribute determination device according to the above 1, wherein the attribute recognition unit recognizes the attribute based on a positional relationship between a plurality of the control points.

4. The attribute recognition unit,
A feature quantity extractor for generating a feature map by extracting a feature quantity for recognizing the attribute from the image,
A feature amount converter for performing a mask process on the feature map,
A discriminator that identifies the attribute based on the masked feature map,
2. The attribute determination according to 1, wherein the feature amount converter performs the mask processing by applying a mask whose degree of decrease in the feature amount increases to a position farther from the control point on the feature map. apparatus.

5. 5. The attribute determination device according to the above 4, wherein the mask has a weight that changes so that the degree of decrease in the characteristic amount has a normal distribution with the control point as the apex.

6. 6. The attribute determination device according to 5, wherein the standard deviation of the normal distribution varies depending on the position of the control point in the skeleton of the person.

7. 7. The attribute determination device according to 5 or 6, wherein the standard deviation of the normal distribution differs depending on the attribute identified by the discriminator.

8. 8. The attribute determination device according to any one of 1 to 7 above, wherein the attribute of the person is at least one of the age, age group, sex, obesity degree, clothing, worn article, and portable article of the person. ..

9. The attribute determination device according to any one of 1 to 8 above,
An attribute determination system, comprising: an image capturing unit that captures an image of a person from above and outputs the image data to the attribute determination device.

10. 10. The attribute determination system as described in 9 above, wherein the image acquisition unit includes an RGB camera.

11. 11. The attribute determination system according to 9 or 10, wherein the image acquisition unit includes a thermal camera or a depth camera.

12. 12. The attribute determination system according to any one of 9 to 11 above, wherein the image acquisition unit includes an infrared camera or a monochrome camera.

13. A position detecting step of detecting the position of a control point, which is a point defining the skeleton of the person, from an image obtained by photographing the person from above,
An attribute recognizing step of recognizing an attribute of the person by using a detection result of the position detecting section.

14. 14. The attribute determining method according to the above 13, wherein in the attribute recognition step, the attribute is recognized based on a time-series variation of the position of the control point.

15. 14. The attribute determination method according to the above 13, wherein the attribute recognition step recognizes the attribute based on a positional relationship between a plurality of the control points.

16. The attribute recognition step,
A feature amount extracting step of generating a feature map by extracting a feature amount for recognizing the attribute from the image,
A feature amount conversion step of performing a mask process on the feature map,
An identification step of identifying the attribute based on the feature map after the mask processing,
14. In the feature amount conversion step, the mask processing is performed by applying a mask whose degree of decrease in the feature amount increases to a position farther from the control point, to perform the mask processing. Method.

17. A program for causing a computer to execute the attribute determination method according to any one of 13 to 16 above.

18. A computer-readable recording medium in which the program described in 17 is recorded.

Although the embodiment of the present invention has been described above, the scope of the present invention is not limited to this, and the present invention can be implemented by being expanded or modified without departing from the gist of the invention.

INDUSTRIAL APPLICATION This invention can be utilized for the apparatus which determines a person's attribute based on the image which image|photographed the person from above.

1 Attribute determination system 2 Image acquisition unit 3 Attribute determination device 16 Position detection unit 17 Attribute recognition unit 17' Attribute recognition unit 17a Feature amount extractor 17b Feature amount converter 17c Discriminator

Claims (18)

A position detection unit that detects the position of a control point, which is a point that defines the skeleton of the person, from an image obtained by photographing the person from above,
An attribute determination device, comprising: an attribute recognition unit that recognizes an attribute of the person by using a detection result of the position detection unit. The attribute determination device according to claim 1, wherein the attribute recognition unit recognizes the attribute based on a time-series variation of the position of the control point. The attribute determination device according to claim 1, wherein the attribute recognition unit recognizes the attribute based on a positional relationship between the plurality of control points. The attribute recognition unit,
A feature quantity extractor for generating a feature map by extracting a feature quantity for recognizing the attribute from the image,
A feature amount converter for performing a mask process on the feature map,
A discriminator that identifies the attribute based on the masked feature map,
2. The attribute according to claim 1, wherein the feature quantity converter performs the mask processing by applying a mask whose degree of decrease in the feature quantity increases to a position farther from the control point to the feature map. Decision device. The attribute determination device according to claim 4, wherein the mask has a weight that changes so that the degree of decrease in the characteristic amount has a normal distribution with the control point as the apex. The attribute determination device according to claim 5, wherein the standard deviation of the normal distribution differs depending on the position of the control point in the skeleton of the person. 7. The attribute determination device according to claim 5, wherein the standard deviation of the normal distribution differs depending on the attribute identified by the discriminator. The attribute determination according to any one of claims 1 to 7, wherein the attribute of the person is at least one of age, age group, sex, obesity degree, clothing, wearable article, and portable article of the person. apparatus. An attribute determination device according to any one of claims 1 to 8,
An attribute determination system, comprising: an image capturing unit that captures an image of a person from above and outputs the image data to the attribute determination device. The attribute determination system according to claim 9, wherein the image acquisition unit includes an RGB camera. The attribute determination system according to claim 9, wherein the image acquisition unit includes a thermal camera or a depth camera. The attribute determination system according to claim 9, wherein the image acquisition unit includes an infrared camera or a monochrome camera. A position detecting step of detecting the position of a control point, which is a point defining the skeleton of the person, from an image obtained by photographing the person from above,
An attribute recognizing step of recognizing an attribute of the person by using a detection result of the position detecting section. 14. The attribute determining method according to claim 13, wherein in the attribute recognizing step, the attribute is recognized based on a time-series variation of the position of the control point. 14. The attribute determining method according to claim 13, wherein in the attribute recognizing step, the attribute is recognized based on a positional relationship between the plurality of control points. The attribute recognition step,
A feature amount extracting step of generating a feature map by extracting a feature amount for recognizing the attribute from the image,
A feature amount conversion step of performing a mask process on the feature map,
An identification step of identifying the attribute based on the feature map after the mask processing,
14. The attribute according to claim 13, wherein, in the feature amount conversion step, the mask processing is performed by applying a mask whose degree of decrease in the feature amount increases to a position farther from the control point to the feature map. How to decide. A program for causing a computer to execute the attribute determination method according to any one of claims 13 to 16. A computer-readable recording medium in which the program according to claim 17 is recorded.

Images