JP2019175093A - Apparatus, method and program for estimation, and apparatus, method and program for learning - Google Patents

Abstract

To provide an apparatus for estimation more suited for estimating a feature point, position or movement of an object concerned.SOLUTION: An apparatus for estimation comprises a compression processing section 23 that compresses each pixel value in an area of an object concerned reflected in an image into a first tonal range out of an overall tonal range expressing the image and an estimating section 24 that performs image analysis using first discriminator model Dm1 after learning on the image on which the compression processing has been done and estimates a feature point, position or movement of the object concerned. The first discriminator model Dm1 has been processed of learning using first learning data Dt1 that a teacher image on which the compression processing has been done is associated with the feature point, position or movement of the object concerned reflected in the teacher image.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to an estimation device, an estimation method, an estimation program, a learning device, a learning method, and a learning program.

  In recent years, there has been a demand for an image recognition technique for recognizing features such as a joint position, posture, or motion of a person shown in the acquired image from the acquired image.

  Against this background, image recognition technology using machine learning has attracted attention. In this type of image recognition technology, the classifier is subjected to machine learning using a teacher image as learning data, thereby causing the classifier to understand the characteristics of the probability distribution latent in the image data. As a result, the learned classifier can identify the image pattern only by inputting the pixel value information of the image.

  For example, Non-Patent Document 1 discloses an image recognition technique for estimating a human posture using a convolutional neural network as a discriminator.

Alexander Toshev, et al. "Deep Pose: Human Pose Estimation via Deep Neural Networks", in CVPR, 2014, ("URL: http://www.cv-foundation.org/openaccess/content_cvpr_2014/papers/Toshev_DeepPose_Human_Pose_2014_CVPR_paper.pdf" )

  By the way, in a discriminator such as a general convolution neural network, color variations (for example, a person's clothes) are excessive when identifying the overall characteristics of the target object (for example, a person's joint position or posture). This is data, and is likely to cause a deterioration in the identification accuracy of the classifier and a learning efficiency of the classifier. Therefore, for example, when learning is performed by the classifier, if an image with different clothes or the like is used as it is as learning data, it takes an enormous amount of learning time until the posture of the person can be accurately identified.

  The present disclosure has been made in view of the above-described problems, and an estimation device, an estimation method, an estimation program, a learning device, a learning method, and a learning program that can suppress a learning time necessary for learning. The purpose is to provide.

The main present disclosure for solving the above-described problems is as follows.
A compression processing unit that compresses each pixel value in the region of the target object shown in the image within a first gradation range of all gradation areas representing the image;
An estimation unit that performs image analysis using the learned first discriminator model on the image subjected to the compression processing, and estimates a feature point, posture, or motion of the target object;
With
The first discriminator model uses the first learning data in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, Is an estimation device.

In other aspects,
A compression processing unit that compresses each pixel value in the region of the target object shown in the teacher image within a first gradation range of all gradation regions representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other A first learning processing unit for performing processing;
It is a learning apparatus provided with.

In other aspects,
Compressing each pixel value in the area of the target object shown in the image within a first gradation range of all gradation areas representing the image;
An estimation method for performing an image analysis using a learned first discriminator model on the image subjected to the compression processing to estimate a feature point, posture, or motion of the target object,
The first discriminator model uses the first learning data in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, This is an estimation method.

In other aspects,
Compressing each pixel value in the area of the target object shown in the teacher image within a first gradation range of all gradation areas representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other Process,
Is a learning method comprising

In other aspects,
On the computer,
A process of compressing each pixel value in the area of the target object shown in the image within a first gradation range of all gradation areas representing the image;
A process of performing image analysis using the learned first discriminator model on the image subjected to the compression process, and estimating a feature point, posture, or motion of the target object;
An estimation program for executing
The first discriminator model uses the first learning data in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, This is an estimation program.

In other aspects,
On the computer,
A process of compressing each pixel value in the area of the target object shown in the teacher image within a first gradation range of all gradation areas representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other Processing to perform processing,
It is a learning program that executes.

  According to the present disclosure, it is possible to suppress the learning time required for learning.

1 is a block diagram showing the overall configuration of an image recognition system according to a first embodiment. Diagram showing the structure of a general convolutional neural network The figure which shows an example of the hardware constitutions of the learning apparatus and estimation apparatus which concern on 1st Embodiment. The figure which shows an example of the human area | region in the image which the area | region detection part which concerns on 1st Embodiment detects. The flowchart which shows an example of the learning process with respect to the 2nd discriminator which concerns on 1st Embodiment. The figure which shows an example of the feature point which the estimation part which concerns on 1st Embodiment estimates, and an attitude | position The flowchart which shows an example of the learning process with respect to the 1st discriminator which concerns on 1st Embodiment. The flowchart which shows an example of the process performed when the estimation apparatus which concerns on 1st Embodiment identifies an image. The figure explaining the structure of the compression process part which concerns on 2nd Embodiment

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the appended drawings. In the present specification and drawings, components having substantially the same function are denoted by the same reference numerals, and redundant description is omitted.

(First embodiment)
[Overall configuration of image recognition system]
Hereinafter, an example of the overall configuration of the image recognition system U according to the first embodiment will be described with reference to FIGS.

  FIG. 1 is a block diagram showing an overall configuration of an image recognition system U according to the present embodiment. In addition, the arrow in FIG. 1 represents the flow of the process of each function, and the flow of data D1-D5.

  The image recognition system U according to the present embodiment performs, for example, image processing on the image data generated by the imaging apparatus, and the joint position, posture, action, etc. of a person reflected in the image (hereinafter referred to as “person's posture, etc.”). Is abbreviated as “)”.

  The image recognition system U according to the present embodiment includes a first discriminator model Dm1 (hereinafter abbreviated as “first discriminator Dm1”) and a second discriminator model Dm2 (hereinafter “second discriminator”). The posture of the person using the learning device 10 that performs learning processing on the learning device 10 and the first discriminator Dm1 and the second discriminator Dm2 that have been subjected to the learning processing by the learning device 10 The estimation apparatus 20 which estimates etc. is provided.

  In the image recognition system U according to the present embodiment, first, after the learning device 10 performs machine learning of the second classifier Dm2 in the second learning processing unit 11 (phase T1), the second Using the discriminator Dm2, the first learning processing unit 14 performs machine learning of the first discriminator Dm1 (phase T2). Then, the estimation device 20 acquires model data relating to the learned second discriminator Dm2 and the learned first discriminator Dm1 from the learning device 10, and executes discrimination processing (phase T3). Each phase T1, T2, T3 is typically executed separately.

  The learning device 10 includes a second learning processing unit 11 that executes the learning process of the phase T1, an area detection unit 12, a compression processing unit 13, and a first learning processing unit 14 that execute the learning process of the phase T2. I have.

  In addition, the estimation device 20 includes an input unit 21 that acquires image data generated by an imaging device or the like, a region detection unit 22 that detects a region in which the target object appears in the image using the learned second discriminator Dm2, A compression processing unit 23 that compresses the pixel value of each pixel region in a region in which the target object is imaged to a predetermined gradation range, an estimation unit 24 that estimates the posture of a person using the learned first discriminator Dm1, and The output unit 25 is provided.

  More preferably, the first discriminator Dm1 and the second discriminator Dm2 are convolutional neural networks having high discrimination performance and robustness against image changes. The model data of the first discriminator Dm1 and the model data of the second discriminator Dm2 shown in FIG. 1 include, for example, data related to the structure of the input layer, intermediate layer, and output layer of the convolutional neural network, and the convolution. It includes data on weighting factors and biases in the neural network.

  FIG. 2 is a diagram illustrating a configuration of a general convolutional neural network.

  The convolutional neural network includes a feature extraction unit Ms and an identification unit Mt. The estimation unit Ms performs a process of extracting an image feature from an input image, and the identification unit Mt identifies a target object from the image feature. Apply processing.

  The feature extraction unit Ms is configured by hierarchically connecting a plurality of feature quantity extraction layers Ms1, Ms2,. Each of the feature quantity extraction layers Ms1, Ms2,... Includes a convolution layer, an activation layer, and a pooling layer.

  The first feature amount extraction layer Ms1 scans an input image for each predetermined size by raster scanning. The feature amount extraction layer Ms1 extracts feature amounts included in the input image by performing feature amount extraction processing on the scanned data using a convolution layer, an activation layer, and a pooling layer. The first layer feature quantity extraction layer Ms1 extracts relatively simple single feature quantities such as a linear feature quantity extending in the horizontal direction and a linear feature quantity extending in the oblique direction.

  The second feature amount extraction layer Ms2 scans an image (also referred to as a feature map) input from the previous feature amount extraction layer Ms1 for each predetermined size by, for example, raster scanning. Then, the feature amount extraction layer Ms2 similarly extracts the feature amount included in the input image by performing the feature amount extraction processing by the convolution layer, the activation layer, and the pooling layer on the scanned data. The second-layer feature quantity extraction layer Ms2 is integrated with consideration of the positional relationship of a plurality of feature quantities extracted by the first-layer feature quantity extraction layer Ms1, so that a higher-dimensional composite is obtained. Feature quantities are extracted.

  The second and subsequent feature quantity extraction layers (not shown) perform the same processing as the second feature quantity extraction layer Ms2. Then, the output of the feature amount extraction layer of the final layer (each value in the map of the plurality of feature maps) is input to the identification unit Mt.

  The identification unit Mt is configured by, for example, a multilayer perceptron in which a plurality of Fully Connected layers are hierarchically connected. The total coupling layer on the input side of the identification unit Mt is fully coupled to each value in the map of the plurality of feature maps acquired from the feature extraction unit Ms, and the product-sum operation is performed while varying the weighting coefficient for each value. Go and output. The all coupled layers in the next layer of the identification unit Mt are fully coupled to the values output by the elements in the all coupled layers in the previous layer, and the product-sum operation is performed while varying the weighting coefficient for each value.

  For example, the identification unit Mt performs a process of applying a softmax function or the like to the output value from each element of the output layer of the multilayer perceptron, and among the plurality of classes, the product corresponding to the class to be identified is multiplied. The identification result is output so that the value of the result of the sum operation becomes large.

  For example, the convolutional neural network is subjected to a learning process using a teacher image with a correct answer class, whereby network parameters (for example, weighting factors and biases of convolutional layers and weighting factors and biases of all connection layers). Are adjusted and function as described above.

  Note that the algorithm of the arithmetic processing in the convolutional neural network is the same as a known method (for example, see Non-Patent Document 1), and thus description thereof is omitted here.

  In order to improve the identification accuracy of such a general convolutional neural network, the convolutional neural network becomes larger as a whole, and the amount of learning for optimizing the convolutional neural network also becomes enormous.

  From this point of view, the image processing system U according to the present embodiment has learned the second classifier Dm2 that detects a region such as a person separately from the first classifier Dm1 that estimates the posture of a person or the like. A region detector 22 for detecting a region such as a person using the second discriminator Dm2, a compression processor 23 for compressing pixel values of each pixel region in the human region R1, and a learned first discriminator The estimation unit 24 that estimates the posture of the person using Dm1 is sequentially executed.

  However, the convolutional neural network constituting the first discriminator Dm1 and the second discriminator Dm2 is not limited to the structure shown in FIG. 2, and various known structures may be applied. For example, as a convolutional neural network constituting the second discriminator Dm2 used for segmentation, a convolutional layer having the same size as the input image may be used as the discriminating unit Mt instead of the multilayer perceptron.

  Further, the first discriminator Dm1 and the second discriminator Dm2 are not limited to the convolutional neural network, but may be an SVM (Support Vector Machine), a Bayes discriminator, or other discriminators. Further, the first discriminator Dm1 and the second discriminator Dm2 may be configured by different types of discriminators. In addition, as the first discriminator Dm1 and the second discriminator Dm2, an ensemble model may be used, or a plurality of types of discriminators may be combined. It may be configured in combination with a pre-processing unit such as a color division process.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of the learning device 10 and the estimation device 20.

  Each of the learning device 10 and the estimation device 20 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an external storage device (for example, a flash memory) 104 as main components. And a computer including the communication interface 105 and the like.

  Each function which the learning apparatus 10 and the estimation apparatus 20 have is implement | achieved, for example, when CPU101 refers to the control program (for example, processing program) and various data which were memorize | stored in ROM102, RAM103, the external storage device 104 grade | etc.,. However, some or all of the functions may be realized by processing by a DSP (Digital Signal Processor) instead of or by processing by the CPU. Similarly, some or all of the functions may be realized by processing by a dedicated hardware circuit instead of or together with processing by software. Note that the learning device 10 and the estimation device 20 are communicatively connected via the communication interface 105 of each other.

[Details of Configuration of Estimation Device and Learning Device]
Next, with reference to FIGS. 4 to 8, each configuration of the estimation device 20 and the learning device 10 according to the present embodiment will be described in detail. Hereinafter, for convenience of explanation, the configuration of the estimation device 20 will be mainly described.

<About the configuration of the input unit 21>
The input unit 21 acquires, for example, image data D1 generated by the imaging device (hereinafter abbreviated as “input image D1”) from the imaging device. Further, the input unit 21 may be configured to acquire image data D1 stored in the external storage device 104 or image data D1 provided via an Internet line or the like.

  Note that the input unit 21 may perform preprocessing such as processing for converting an image into a predetermined size, processing for converting to an aspect ratio, or color division processing.

<Regarding Configuration of Area Detection Unit 22>
The region detection unit 22 acquires the input image D1 from the input unit 21. Then, the region detection unit 22 performs image analysis using the learned second discriminator Dm2 on the input image D1, and represents a region in which the target object appears in the image (here, a region in which a person appears). (Hereinafter referred to as “human region”).

  FIG. 4 is a diagram illustrating an example of a human region in an image detected by the region detection unit 22.

  In FIG. 4, Rall represents the entire image region of the image, R1 represents a human region in the image, and R2 represents a region other than the human region R1 in the image (hereinafter referred to as “surrounding region”).

  The convolutional neural network that constitutes the second discriminator Dm2 is, for example, a pixel area that outputs a class (here, a person class or a class other than a person) to which each pixel area in the image corresponds to an input image. Each output element.

  The second classifier Dm2 uses the second learning processing unit 11 to identify and output the pixel area corresponding to the human area R1 and the pixel area corresponding to the surrounding area R2 in the input image. A learning process has been performed.

  FIG. 5 is a flowchart illustrating an example of a learning process (learning phase T1) for the second discriminator Dm2.

  In step S11, the learning device 10 first generates second learning data Dt2. The learning device 10 detects a human region R1 in the teacher image using, for example, a background difference method, template matching, or HOG (Histograms of Oriented Gradients) feature amount, and associates the human region R1 with the teacher image. Thus, the second learning data Dt2 is generated.

  The second learning data Dt2 is a data set in which a teacher image and correct data related to the human region R1 in the teacher image are associated with each other. The second learning data Dt2 has a plurality of such data sets.

  In step S12, the learning device 10 (second learning processing unit 11) performs learning processing on the second discriminator Dm2 using the second learning data Dt2.

  In step S12, for example, the second learning processing unit 11 inputs the teacher image of the second learning data Dt2 to the input element of the second discriminator Dm2, and outputs the second discriminator Dm2. Correct data related to the human region R1 of the second learning data Dt2 is set in the element (for example, “1” is set for the human region R1 and “0” is set for other than the human region R1), and the second discriminator. Dm2 network parameters (weighting factor, bias, etc.) are optimized. At this time, the second learning processing unit 11 uses the network parameter of the second discriminator Dm2 so that the loss function is minimized by, for example, a known error back propagation method using the cross entropy as the loss function. (Weighting factors, bias, etc.) are optimized.

  In step S <b> 13, the learning device 10 stores the network parameters of the second discriminator Dm <b> 2 that has undergone the learning process, for example, in the external storage device 104 and inputs the network parameters to the region detection unit 12 and the region detection unit 22. Then, a series of processing ends.

  In this way, the region detection unit 22 performs image analysis (for example, forward propagation processing of a convolutional neural network) on the input image D1 using the second discriminator Dm2 subjected to the learning processing, and performs input. A human region R1 is detected from the image D1.

<Configuration of Compression Processing Unit 23>
The compression processing unit 23 acquires the input image D1 from the region detection unit 22 and data D2 related to the human region R1 detected in the input image D1. Then, the compression processing unit 23 compresses each pixel value of the human region R1 in the input image D1 within a predetermined gradation range in the entire gradation region expressing the image. That is, the compression processing unit 23 reduces the color variation of the human region R1.

  The inventors of the present invention have intensively studied from the viewpoints of improving identification accuracy, downsizing the classifier, improving learning efficiency, etc., and when color variations such as clothes identify a person's posture, etc. Obtaining the knowledge that the data is excessive and easily deteriorates the discrimination accuracy of the discriminator and the learning efficiency of the discriminator, the compression processing unit 23 has been introduced.

  For example, the compression processing unit 23 sets the pixel value (in this case, any value from 0 to 255) in the human region R1 expressed by 256 gradations within a gradation range 200 to 255 of about 1/5. Compress. The compression processing performed by the compression processing unit 23 is typically processing for reducing the difference in pixel values while maintaining the magnitude relationship between the pixel values between the pixel regions. As a result, it is possible to grasp the positional relationship of the main part (for example, each part of the human body) of the human region R1 necessary for identifying the posture of the person, and the excessive color information included in the human region R1. Compress.

  At this time, the gradation range after compression is more preferably a range narrower than the gradation range of the upper third of the entire gradation range (for example, 200 to 255 corresponding to the white side). (Gradation range) or a range narrower than the lower one-third gradation range (for example, 0 to 50 gradation range corresponding to the black side). As a result, the pixel value in the human region R1 is separated from the pixel value included in the surrounding region R2, and in the estimation unit 24 in the subsequent stage, in addition to the image of the human region R1, The image in the region R2 can also be used effectively.

The compression processing unit 23 performs a compression process on the pixel value of the human area R1 using, for example, the following equation (1).
Y = (X−x_min) × (max−min) / (x_max−x_min) + min (1)
(However, X: Pixel value of input pixel area, Y: Output pixel value after compression,
max: maximum pixel value (255 here) that can be expressed in the gradation range after compression,
min: Minimum pixel value (200 here) that can be expressed in the gradation range after compression,
x_max: the maximum pixel value (here 255) in all gradation ranges that can be expressed by the image before compression,
x_min: Minimum pixel value of all gradation regions that can be represented by the image before compression (here, 0))

  The compression processing unit 23 inputs the pixel value of each pixel area of the human area R1 to X in Expression (1), and converts it into a compressed pixel value Y. As a result, the pixel values in the human region R1 expressed in 256 gradations can be compressed within the gradation range 200 to 255 while maintaining the magnitude relationship of the pixel values between the pixel regions.

Note that the compression processing of the compression processing unit 23 may use a method other than the equation (1), and for example, the compression processing may be performed using the following equation (2).
Y = {(X−μ) / σ} × (max−min) / 2 + (min + max) / 2 (2)
(However, X: Pixel value of input pixel area, Y: Output pixel value after compression,
μ: average value of pixel values in the human region R1,
σ: standard deviation of the pixel value of the human region R1,
max: maximum pixel value (255 here) that can be expressed in the gradation range after compression,
min: Minimum pixel value (200 here) that can be expressed in the gradation range after compression

  According to the equation (2), after performing normalization processing in which the average value of the pixel values in the human region R1 is 0 and the variance is 1, the pixel values in the human region R1 expressed in 256 gradations The pixel values can be compressed within the gradation range 200 to 255 while maintaining the magnitude relationship between the pixel values.

  Note that the compression processing unit 23 according to the present embodiment is configured not to perform the above-described compression processing on the image of the surrounding area R2 in order to avoid excessive reduction of the color information of the surrounding area R2. Similarly, the image may be subjected to the above-described compression processing (see, for example, a second embodiment described later).

<About the structure of the estimation part 24>
The estimation unit 24 acquires image data D3 obtained by performing compression processing on the input image D1 by the compression processing unit 23. Then, the estimation unit 24 performs image analysis using the learned first discriminator Dm1, and estimates the feature point, posture, or motion of the target object (here, a person).

  FIG. 6 is a diagram illustrating an example of a person's feature points and postures estimated by the estimation unit 24. FIG. 6 corresponds to FIG.

  As the feature points, postures, or movements estimated by the estimation unit 24, typically, the joint position of the human body, the position of each part of the human body (for example, the position of the head, the position of the foot), and the posture of the human body The type (for example, the state of standing up, the state of bending forward), the type of movement of the human body (for example, the state of trying to pick up an object, the state of reading, etc.), or temporal changes thereof. The data format output by the estimation unit 24 is arbitrary, such as a type format, a coordinate format, or a relative position between parts.

  In FIG. 6, as an example of the feature points and postures estimated by the estimation unit 24, the right ankle p0, the right knee p1, the right hip p2, the left hip p3, the left knee p4, the left ankle p5, the right wrist p6, and the right elbow p7. , The right shoulder p8, the left shoulder p9, the left elbow p10, the left wrist p11, the throat p12, and the head joint position of the human head p13, and the posture class of the human body such as “posture class No: 1 (for example, standing)” Is shown.

  The convolutional neural network constituting the first discriminator Dm1 includes, for example, an output element that outputs the coordinates (X coordinate, Y coordinate) of the joint positions p0 to p13 of the human body with respect to the input image, and the human body It includes an output element that outputs a posture class of

  As shown in FIG. 6, the first discriminator Dm1 performs the first learning so as to output the coordinates of the joint positions p0 to p13 of the human body and the posture class of the human body based on the input image. A learning process is performed by the processing unit 14.

  FIG. 7 is a flowchart illustrating an example of a learning process (learning phase T2) for the first discriminator Dm1.

  In step S21, the learning device 10 first generates first learning data Dt1. For example, the learning device 10 sets the posture of the person (in this case, the coordinates of the joint positions p0 to p13 of the human body and the posture class of the human body) with respect to the teacher image in which the person is reflected by manual input. Thus, the first learning data Dt1 is generated.

  The first learning data Dt1 includes the teacher image and correct data related to the posture of the person shown in the teacher image (for example, the coordinates of the joint positions p0 to p13 of the human body and the posture class of the human body). Is the associated data set. The first learning data Dt1 has a plurality of such data sets.

  In step S22, the learning device 10 (region detection unit 12) detects the human region R1 of the image using the learned second discriminator Dm2.

  Note that the region detection unit 12 that executes step S22 typically has the same configuration as the region detection unit 22 of the estimation device 20. That is, in this step S22, the region detection unit 12 performs image analysis (for example, forward propagation processing of a convolutional neural network) using the learned second discriminator Dm2, and detects a human region R1 in the image. To do.

  In step S23, the learning device 10 (compression processor 13) performs a process of compressing the pixel value of each pixel area of the human area R1 estimated in step S22 within a predetermined gradation range.

  The compression processing unit 13 that executes this step S23 typically has the same configuration as the compression processing unit 23 of the estimation device 20. That is, in this step S23, the compression processing unit 13 uses the pixel value (here, 0 to 255) of each pixel region of the human region R1 expressed in 256 gradations, and the magnitude relationship of the pixel values between the pixel regions. Is maintained within the gradation range 200 to 255.

  In step S24, the learning device 10 (first learning processing unit 14) performs learning processing on the first discriminator Dm1 using the first learning data Dt1.

  In this step S24, for example, the first learning processing unit 14 inputs a teacher image to the input element of the first discriminator Dm1, and also outputs it to the output element of the first discriminator Dm1 as correct answer data. Set the posture of the person shown in the image (for example, set correct values of the coordinates of the joint positions p0 to p13 of the human body, set the correct class of the posture class of the human body to “1”, and other classes Is set to “0”), the network parameters (weight coefficient, bias, etc.) of the first discriminator Dm1 are optimized. At this time, the first learning processing unit 14 uses the network parameter of the first discriminator Dm1 so that the loss function is minimized by, for example, a known error back propagation method using the cross entropy as the loss function. (Weighting factors, bias, etc.) are optimized.

  In step S25, the learning device 10 stores the network parameter of the first discriminator Dm1 adjusted by the learning process in the storage unit (for example, the external storage device 104) and inputs it to the estimation unit 24 of the estimation device 20. To do.

  The estimation unit 24 uses the first discriminator Dm1 subjected to learning processing in this way to perform image analysis (for example, forward propagation processing of a convolutional neural network) on the image D3 input from the compression processing unit 23. ) To estimate the posture or the like of the person shown in the image D3.

  The image D3 input to the first discriminator Dm1 by the estimation unit 24 may be only the image of the human region R1, but more preferably includes at least a part of the image of the surrounding region R2. Accordingly, since the posture of the person can be estimated from the relationship with the object or the like in the surrounding region R2, the identification accuracy of the posture of the person can be further improved.

<About the configuration of the output unit 25>
The output unit 25 acquires data D4 such as the posture of the person output from the estimation unit 24. Then, the output unit 25 processes the data D4 into a predetermined image format, and outputs the processed data D5 to a display device or the like.

  At this time, for example, the output unit 25 superimposes and displays the positions of the joint positions p0 to p13 estimated by the estimation unit 24 and the class related to the posture of the person with the highest class matching degree with the input image D1. Output to a display device or the like.

[Operation of estimation device]
FIG. 8 is a flowchart illustrating an example of processing executed when the estimating apparatus 20 estimates the posture of a person shown in an image.

  In step S31, the estimation apparatus 20 (input unit 21) acquires image data D1 and performs preprocessing such as processing an image of the image data into a predetermined shape.

  In step S32, the estimation device 20 (region detection unit 22) performs image analysis using the second discriminator Dm2 on the input image acquired in step S31, and a person in which a person appears in the input image D1 A region R1 is detected.

  In step S33, the estimation device 20 (compression processor 23) uses the above-described equation (1) and the like to calculate the pixel value of each pixel area in the human area R1 of the input image estimated in step S32 to a predetermined floor. Compress within key range.

  In step S34, the estimating apparatus 20 (estimating unit 24) performs image analysis using the first discriminator Dm1 on the image D3 subjected to the compression processing in step S33, and the person's character is included in the input image. Estimate posture.

  In step S35, the estimation device 20 (output unit 25) converts the posture of the person generated in step S34 into a predetermined image format and outputs the image to a display device or the like.

[effect]
As described above, the estimation apparatus 20 according to the present embodiment uses the pixel values of each pixel region in the region (the human region R1 according to the present embodiment) in which the target object is shown in the image D1 as the whole floor that represents the image D1. The compression processing unit 23 that compresses within the first gradation range (for example, 200 to 255) in the adjustment range (for example, 0 to 255) and the image D3 that has been subjected to the compression processing have been learned. An estimation unit 24 that performs image analysis using the first discriminator Dm1 and estimates the posture of the person and the like.

  Therefore, according to the estimation apparatus 20 according to the present embodiment, the first discriminator Dm1 that is less affected by color variations such as clothes and is more suitable for identifying the feature point, posture, action, or the like of the target object. It is possible to build In other words, this makes it possible to reduce the size of the first discriminator Dm1, improve the discrimination accuracy by the first discriminator Dm1, improve the learning efficiency of the first discriminator Dm1, and the like. This also reduces the amount of learning data for performing the learning process on the first discriminator Dm1.

  In addition, the estimation device 20 according to the present embodiment separately includes a second discriminator Dm2 for the region detection unit 22 and a first discriminator Dm1 for the estimation unit 24, and the second discriminator. The posture or the like of the target object is identified by two-stage image analysis using Dm2 and the first classifier Dm1. This makes it possible to maximize the robustness against image diversity, which is a characteristic of machine learning. This also improves the detection accuracy of the human region R1, and as a result, the identification accuracy of the first discriminator Dm1 and the learning efficiency of the first discriminator Dm1 can be further improved.

  Further, in the estimation apparatus 20 according to the present embodiment, the compression processing unit 23 calculates the pixel value of each pixel region in the region (the human region R1 according to the present embodiment) in which the target object in the image D1 is reflected in the entire gradation region. Are compressed to the upper gradation range or the lower gradation range. As a result, the identification accuracy of the first classifier Dm1 can be further improved, and the learning efficiency of the first classifier Dm1 can be improved. In addition, this makes it possible to increase the contrast between the human region R1 and the surrounding region R2, thereby further improving the identification accuracy such as the posture of the target object.

  In the above embodiment, the joint position, posture, or movement of the entire person is shown as an example of the target object identified by the first classifier Dm1. However, the target object identified by the first classifier Dm1 of the present invention may be a specific part of a person (for example, a head or an arm). In that case, the region detection unit 22 may detect a region of a specific part of the person shown in the image, and the compression processing unit 23 may set the target region to be subjected to compression processing only to the specific part of the person. Good. For example, when the target object to be identified is the head, the target region on which the compression process is performed may be the hair portion. Further, for example, when the target object to be identified is an arm part, a target area to be subjected to compression processing may be a clothing part.

(Second Embodiment)
The image recognition system U according to the present embodiment is different from the first embodiment in the configuration of the compression processing unit 23 (and the compression processing unit 13).

  FIG. 9 is a diagram illustrating the configuration of the compression processing unit 23 according to the present embodiment.

  FIG. 9 shows the distribution of the pixel values R1a in the human region R1 (appearance frequency for each pixel value) in the entire gradation region (here, 0 to 255) and the distribution of the pixel values R2a in the surrounding region R2 (for each pixel value). Frequency of occurrence) schematically.

  In the compression processing unit 23 according to the present embodiment, each pixel value R1a in the human region R1 is narrower than the upper one-side gradation range of the entire gradation region (here, 0 to 255). While compressing to the range R1b (here, the gradation range of 200 to 255 corresponding to the white side), the pixel value R2a in the surrounding region R2 is set to the gradation of the lower third of the entire gradation region. Compression is made to a range R2b (here, a gradation range of 0 to 50 corresponding to the black side) that is narrower than the range. That is, in the compression processing unit 23 according to the present embodiment, each pixel value R1a in the human region R1 and each pixel value R1b in the surrounding region R2 in the image are separated from each other in all gradation regions expressing the image. So that

  As described above, according to the image recognition system U according to the present embodiment, the contrast between the pixel value in the human region R1 and the pixel value in the surrounding region R2 can be increased, whereby the first discriminator Dm1. It is possible to further improve the identification accuracy.

(Other embodiments)
The present invention is not limited to the above embodiment, and various modifications can be considered.

  In the above-described embodiment, the joint position, posture, or movement of a person is shown as an example of “identification target” in the image recognition system U. However, the image recognition system U according to the present invention is a feature of an arbitrary object reflected in an image. The present invention can be applied to estimate a point, posture, or motion, and can be applied to, for example, estimate the posture of an animal, a vehicle, a figurine, or the like. Further, the image recognition system U according to the present invention can be applied to image data of an illustration image instead of the image data generated by the imaging apparatus.

  Moreover, in the said embodiment, the aspect comprised by the convolution neural network as an example of the area | region detection part 22 was shown. However, the method by which the region detection unit 22 detects the human region R1 is not necessarily limited to a method using a discriminator (learning device) such as a convolutional neural network. For example, the region detection unit 22 may detect the human region R1 by using a conventional method such as a background subtraction method, template matching, HOG (Histograms of Oriented Gradients) feature, or SVM (Support Vector Machine).

  In the above-described embodiment, as an example of the compression processing unit 23, a mode in which the pixel values of the human region R1 are uniformly compressed for the three primary colors of RGB is shown. However, the compression processing unit 23 may change the compression rate (that is, the gradation range) for each of RGB.

  Moreover, in the said embodiment, although the learning apparatus 10 and the estimation apparatus 20 showed the aspect implement | achieved by a separate computer, as an aspect implement | achieved by the learning apparatus 10 and the estimation apparatus 20 by one computer, Of course. Further, programs and data read by a computer, data written by the computer, and the like may be distributed and stored in a plurality of computers.

  Specific examples of the present invention have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above.

  At least the following matters will become apparent from the description of this specification and the accompanying drawings.

A compression processing unit 23 that compresses each pixel value in the region of the target object shown in the image within a first gradation range of all gradation areas representing the image;
An estimation unit 24 that performs image analysis using the learned first discriminator model Dm1 on the image subjected to the compression processing, and estimates a feature point, posture, or motion of the target object;
With
The first discriminator model Dm1 uses the first learning data Dt1 in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, An estimation device 20 that has been subjected to learning processing is disclosed.

More preferably, the estimation device 20 performs image analysis using the learned second discriminator model Dm2 on the image, and detects a region of the target object appearing in the image. An area detector,
The second discriminator model Dm2 is subjected to learning processing using second learning data Dt2 in which a teacher image and a region where the target object is reflected in the teacher image are associated.

  In the estimation device 20, the compression processing unit 23 more preferably performs the compression processing so as to reduce the difference in pixel values between the pixel regions.

  In the estimation apparatus 20, the first gradation range is more preferably a range narrower than the upper one-side gradation range of the entire gradation range expressing the image, Or, it is set to a range narrower than the lower one-third gradation range.

  Further, in the estimation device 20, the compression processing unit 23 more preferably further calculates each pixel value in the peripheral region of the target object in the image, out of the entire gradation region expressing the image. Compress to a second gradation range separated from the first gradation range.

  In the estimation device 20, the target object is more preferably a person.

  In the estimation device 20, the first discriminator model Dm1 and the second discriminator model Dm2 are more preferably configured so as to each include a convolutional neural network.

In other aspects,
A compression processing unit 13 that compresses each pixel value in the region of the target object shown in the teacher image within a first gradation range of all gradation regions representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other A first learning processing unit 14 for performing processing;
The learning apparatus 10 provided with this is disclosed.

In other aspects,
Compressing each pixel value in the area of the target object shown in the image within a first gradation range of all gradation areas representing the image;
An estimation method for performing an image analysis using the learned first discriminator model Dm1 on the image subjected to the compression process to estimate a feature point, posture, or motion of the target object. ,
The first discriminator model Dm1 uses the first learning data Dt1 in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, An estimation method in which learning processing is performed is disclosed.

In other aspects,
Compressing each pixel value in the area of the target object shown in the teacher image within a first gradation range of all gradation areas representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other Process,
A learning method comprising:

In other aspects,
On the computer,
A process of compressing each pixel value in the area of the target object shown in the image within a first gradation range of all gradation areas representing the image;
A process of performing image analysis using the learned first discriminator model Dm1 on the image subjected to the compression process, and estimating a feature point, posture, or motion of the target object;
An estimation program for executing
The first discriminator model Dm1 uses the first learning data Dt1 in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, An estimation program that has been subjected to learning processing is disclosed.

In other aspects,
On the computer,
A process of compressing each pixel value in the area of the target object shown in the teacher image within a first gradation range of all gradation areas representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other Processing to perform processing,
A learning program for executing is disclosed.

  The estimation device according to the present disclosure is more suitable for estimating the feature point, posture, or motion of the target object.

U image recognition system 10 learning device 11 second learning processing unit 12 region detection unit 13 compression processing unit 14 first learning processing unit 20 estimation device 21 input unit 22 region detection unit 23 compression processing unit 24 estimation unit 25 output unit Dm1 1st discriminator Dm2 2nd discriminator

Claims (12)

A compression processing unit that compresses each pixel value in the region of the target object shown in the image within a first gradation range of all gradation areas representing the image;
An estimation unit that performs image analysis using the learned first discriminator model on the image subjected to the compression processing, and estimates a feature point, posture, or motion of the target object;
With
The first discriminator model uses the first learning data in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, Has been given,
Estimating device. A region detection unit that performs image analysis using the learned second discriminator model on the image and detects a region of the target object reflected in the image;
The second discriminator model is subjected to learning processing using second learning data in which a teacher image and a region in which the target object is reflected in the teacher image are associated.
The estimation apparatus according to claim 1. The compression processing unit performs the compression processing so as to reduce a difference in pixel values between the pixel regions.
The estimation apparatus according to claim 1 or 2. The first gradation range is a range that is narrower than the upper one-side gradation range of all gradation areas representing the image, or the lower-limit one-third gradation range. Set to a narrower range,
The estimation apparatus as described in any one of Claims 1 thru | or 3. The compression processing unit further includes a second pixel value obtained by separating each pixel value in a peripheral region of the target object in the image from the first gradation range in the entire gradation area representing the image. Compress to the gradation range,
The estimation apparatus as described in any one of Claims 1 thru | or 4. The target object is a person or a predetermined part of a person,
The estimation apparatus as described in any one of Claims 1 thru | or 5. Each of the first classifier model and the second classifier model includes a convolutional neural network.
The estimation apparatus according to claim 2. A compression processing unit that compresses each pixel value in the region of the target object shown in the teacher image within a first gradation range of all gradation regions representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other A first learning processing unit for performing processing;
A learning apparatus comprising: Compressing each pixel value in the area of the target object shown in the image within a first gradation range of all gradation areas representing the image;
An estimation method for performing an image analysis using a learned first discriminator model on the image subjected to the compression processing to estimate a feature point, posture, or motion of the target object,
The first discriminator model uses the first learning data in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, Has been given,
Estimation method. Compressing each pixel value in the area of the target object shown in the teacher image within a first gradation range of all gradation areas representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other Process,
A learning method comprising: On the computer,
A process of compressing each pixel value in the area of the target object shown in the image within a first gradation range of all gradation areas representing the image;
A process of performing image analysis using the learned first discriminator model on the image subjected to the compression process, and estimating a feature point, posture, or motion of the target object;
An estimation program for executing
The first discriminator model uses the first learning data in which the teacher image subjected to the compression processing and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other, Has been given,
Estimation program. On the computer,
A process of compressing each pixel value in the area of the target object shown in the teacher image within a first gradation range of all gradation areas representing the teacher image;
Learning with respect to the first discriminator model using the first learning data in which the compression-processed teacher image and the feature point, posture, or motion of the target object reflected in the teacher image are associated with each other Processing to perform processing,
Learning program to execute

Images