JP2017041218A - System for estimating weight based on face image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To analyze plural face images, and measure a weight based on a correlation relationship between a part which affects a weight and a part which does not affect a weight.SOLUTION: Plural face images are sent to a network NW1 from portable terminals 100-100of users. The pieces of sent face image data are accumulated in an information processor 200 through the Internet network. The pieces of accumulated face image data can be browsed and analyzed on a PC through the Internet network, and analysis is performed about a width between eyes, a nose, a mouth, and a width of a contour of a face, and an angle where plural tangents cross on the contour of the face, in the face. Then, a weight is measured according to the correlation relationship based on the analyzed data, and the measured weight is accumulated in the information processor 200 and displayed on the user's portable terminals 100-100. The PC learns based on the face image analysis, and statistical data in the correlation relationship is modified for improving weight measurement accuracy.SELECTED DRAWING: Figure 1

Description

The present invention relates to a program for estimating the weight of a person from the face image of the person, an information processing apparatus, a network apparatus, and a server apparatus.

A scale is generally used as a device for measuring body weight. As an application of this weight scale, for example, there is a high-function weight scale that can measure a weight and then transmit the value to an external storage device to record a history of weight measurement of the person.

Face recognition technology is generally used to detect a face image from a person image. As an applied technique using this technique, for example, there is a technique capable of estimating the age and sex of a person from the feature of the face image after detecting the face image.

Patent Publication 2004-265431 Patent application 2004-110880

Problems to be solved by the invention

The above scales have the following problems. In order to measure the weight, it is necessary to get on a scale that is stationary at home, so it cannot be used easily from the outside. In addition, a high-performance weight scale having a function capable of recording the weight in an external storage device is more expensive than a general weight scale not having the function. On the other hand, it is difficult to spread.

Furthermore, these problems are required to be solved for the following reasons. According to the Ministry of Internal Affairs and Communications' Public Opinion Survey 1996, “My Health” (44%) is the highest in the percentage of matters of interest in national life, and “Family Health” (36%) is the third. It shows that the public has a high interest in medical care and healthcare. In particular, in recent years, maintaining weight and BMI at normal values has been emphasized from the viewpoint of preventing lifestyle-related diseases and the aesthetic viewpoint of appearance. For this reason, there is a growing need for automatically recording and managing weight regardless of location.

Means for solving the problem

In order to solve the above problems, the present invention applies face recognition technology to estimate the weight of a person from the face image of the person, and automatically stores the weight in combination with an external recording device. Provide enabling software and system configuration.

A user of the present invention operates software on a portable terminal that is already held, such as a smartphone, a personal computer, a portable game machine, or an electronic book reader. The software estimates the weight of the person from the face image, and the estimated weight is automatically recorded and managed on an electronic recording medium.

Effect of the invention

Since the present invention can be used with terminals such as smartphones that are already widely used, it is possible to estimate and record the weight regardless of location. This eliminates the need to ride on a scale that is stationary at home when measuring weight. Further, there is no need to obtain a high-performance weight scale having a function of recording the weight in the external storage device at a high price. As described above, the present invention has an effect of solving the above-described problems.

The system accumulates various human face images and weight estimation results, but it is assumed that the accumulated estimation results include those that could not be estimated correctly. If the user feels that his / her weight has not been correctly estimated by the system, he can offer a correction of the estimated weight by sending the correct value to the system. The system performs machine learning using the corrected value as learning data, thereby improving the estimation accuracy thereafter.

In addition, the user can send his face photo and health status to the system. The system can estimate the person's health state from the face photograph by performing machine learning using the face photograph and the health state as learning data.

  The figure which shows the outline of the composition of the weight measurement system   Block diagram showing the hardware configuration of the client terminal   Block diagram showing hardware configuration of information processing apparatus   Data structure example of user DB   Data structure example of weight DB   The figure which shows the flow until displaying the result of weight estimation   The figure which shows the example of a display of the estimated weight on a client terminal   Example 1 of facial part used for analysis   Schematic diagram of analysis method 1   Example 2 of facial parts used for analysis   Schematic diagram of analysis method 2

The present invention according to claim 1 and claim 2 provides the following system configuration and processing flow.

First, the system configuration will be outlined.

The system requires at least the client terminal 100 owned by the user. The client terminal 100 corresponds to a smartphone, a personal computer, a portable game machine, an electronic book reader, or the like. In any case, the client terminal 100 requires at least a CPU and a memory for executing software and a display for displaying information to the user.

When the client terminal 100 has a high-performance CPU and memory and can execute a complicated image processing program with a large CPU load, a weight estimation program 900 described later is executed inside the client terminal 100. It is possible. In this way, a method that operates only with a single terminal without depending on other terminals is named a “stand-alone application method”. The present invention makes it possible to operate the weight estimation program 900 by a stand-alone application method and display the result to the user.

However, the present invention does not necessarily have to adopt a stand-alone application method. A method that operates by cooperating with other devices via a network is named a “client-server application method”. In the present invention, for example, the weight estimation program 900 is executed inside the information processing apparatus 200 such as a calculation server machine, and the result is displayed on the client terminal 100. In this case, the client terminal 100 needs a communication unit for communicating with the information processing apparatus 200.

In the following, the details when the present invention is realized by the client-server type application method will be described. However, it should be noted that the present invention can also be realized by the stand-alone type application method.

FIG. 1 is a diagram showing a system configuration when the present invention is implemented by a client-server type application system. The system records at least the client terminal 100 owned by the user, the information processing apparatus 200 on which a weight estimation program 900 described later operates, a network NW1 for connecting them to the network, and user information. A user database DB1, a face photo database DB2 for recording face photo information, and a storage STR1 for storing actual photo data.

In FIG. 1, a plurality of client terminals 100 are illustrated, which are denoted as 100_1, 100_2, and 100_N, respectively. The client terminal 100 may be, for example, a smartphone, a tablet terminal, a portable game machine, or an electronic book reader.
Further, all the client terminals 100 are connected to the single information processing apparatus 200 via the network NW1, but a plurality of information processing apparatuses 200 may exist. In this case, any client terminal 100 can be connected to any information processing apparatus 200 via the network NW1. The network NW1 is an arbitrary network for connecting the client terminal 100 and the information processing apparatus 200, and is, for example, the Internet, a wide area Ethernet, a VPN, or a LAN.

FIG. 2 is a block diagram illustrating a hardware configuration of the client terminal 100. The client terminal 100 includes at least a control unit 110, a storage unit 120, a communication unit 130, a calculation unit 140, an input unit 150, and a display unit 160. The control unit 110 controls the operation of each unit of the client terminal 100. The storage unit 120 is a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), or an SSD (Solid State Drive). The communication unit 130 is a wired communication or wireless communication device such as a network adapter having a function for connecting to the information processing device 200 via the network NW1. The arithmetic unit 140 has a function for executing a program, and is a device that controls a CPU (Central Processing Unit), an arithmetic logic unit ALU (Arithmetic Logic Unit), various registers and counters, instruction decoding and execution control. The input unit 150 is an input device such as a mouse, a keyboard, a touch panel, or a camera for a user of the client terminal 100 to operate the client terminal 100 or input information to the client terminal 100. The display unit 160 includes a display panel using a liquid crystal element or an organic EL (electroluminescence) element for visually transmitting information to the user of the client terminal 100, and a drive circuit that drives each element of the display panel. It is a display device.

FIG. 3 is a block diagram illustrating a hardware configuration of the information processing terminal 200. The information processing terminal 200 includes at least a control unit 210, a storage unit 220, a communication unit 230, and a calculation unit 240. The control unit 210 controls the operation of each unit of the information processing terminal 200. The storage unit 220 is a storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), a HDD (Hard Disk Drive), and an SSD (Solid State Drive). The communication unit 230 is a wired communication or wireless communication device such as a network adapter having a function for connecting to the client terminal 100 via the network NW1. The arithmetic unit 240 has a function for executing a program, and is a device that controls a CPU (Central Processing Unit), an arithmetic logic unit ALU (Arithmetic Logic Unit), various registers and counters, instruction decoding and execution control.

FIG. 4 is a diagram illustrating a configuration example of data recorded in the user database DB1. Elements such as user ID, name, and height are recorded for each user. The user ID is a unique character string for distinguishing users. The name is a name registered by the user in the system. Height is the height registered by the user in the system.

FIG. 5 is a diagram illustrating a configuration example of data recorded in the weight database DB2. For each piece of weight data, elements such as weight ID, user ID, face image URL, estimated weight, actual weight, date and time, and health status are recorded. The weight ID is a unique character string for distinguishing weight data. The user ID is a character string indicating which user the weight data is associated with, and is the user ID shown in FIG. The URL of the face image is a URL that represents the actual storage location of the face image. The estimated weight is the weight of the user estimated by a weight estimation program 900 described later. The date and time is the date and time when this data was written in the weight database DB2.

Next, the processing flow will be outlined.

FIG. 6 is a diagram illustrating an example of a flow until the estimated weight is displayed on the display unit 160 of the client terminal 100. The user can transmit from the client terminal 100 to the information processing apparatus 200 the ID for identifying the user, the face photograph, and the correct answer data of the weight. However, here, the correct answer data is data described in claim 1. Moreover, transmission of correct data is not essential. The information processing apparatus 200 searches for user information from the user database DB1 with the received ID. The information processing apparatus 200 extracts a feature amount necessary for weight calculation from the received face photograph. The information processing apparatus 200 estimates the weight by executing a weight estimation program 900 described later. The elements shown in FIG. 5 are written in the information processing apparatus 200 weight database DB2. The estimated weight is returned to the client terminal 100. The client terminal displays the received estimated weight.

FIG. 7 is a diagram illustrating an example of a display screen of the client terminal 100. FIG. 7 illustrates an example in which the client terminal is a smartphone. Reference numeral 150 denotes a display unit of a smartphone, which includes a face photo display unit L001 and an estimated weight display unit L002. In addition, a button Bt4 for starting an action of sending a photo to the information processing apparatus 200, a button Bt1 for selecting a face photograph stored in the smartphone, a camera built in the smartphone, a front camera and a back A button Bt2 for switching cameras may be included.

FIG. 8 is an example 1 of the feature quantity representing the shape of the head, which is used by the weight estimation program 900 for estimating the weight. With your eyes open? Is it the upper or lower eyelid in the inner eye angle? The transition point is called “inner eye corner point”, and the distance between both inner eye corner points is called “distance between both inner eye corner points”. The head side? The outermost point is called “temporal point”, and the distance between both temporal points is called “distance between both temporal points”. The outermost point of the zygomatic arch is called “zygomatic arch point”, and the distance between both zygomatic arch points is called “distance between both zygomatic arch points”. Mandible angle? The point that protrudes to the outermost side is called “jaw angle point”, and the distance between both jaw angle points is called “distance between both jaw angle points”. In addition, the correction shown by Claim 6, Claim 7, and Claim 8 is applied to these distances.

FIG. 9 is a schematic diagram of the weight estimation method. The body weight is estimated by a function using the distance between the bilateral head points, the distance between both zygomatic arch points, and the distance between both jaw corner points as inputs and the weight as an output. Although any function system may be used, FIG. 9 illustrates a linear combination function of each variable as an example. The function includes variable parameters, and the parameters can be learned using the machine learning system according to claim 3.

FIG. 10 is an example 2 of the feature quantity representing the shape of the head, which the weight estimation program 900 uses for weight estimation. The contours at both ends of the jaw angle point are approximated by a straight line, and the angle between the two straight lines is called a “jaw angle angle”. The average value of both jaw angle points is called “average jaw angle”.

FIG. 11 is a schematic diagram of the weight estimation method. There is a correlation between the cosine of the average jaw angle angle and the person's BMI, and the data is plotted in a coordinate space where the horizontal axis is the BMI and the vertical axis is the cosine of the average jaw angle angle. It can be approximated by a linear function. The BMI is estimated from the average jaw angle point angle, and the weight is estimated by multiplying the estimated BMI by the square of the height. The slope of the linear function can be learned using the machine learning system according to claim 3.

Claims (8)

A machine learning system for estimating a person's weight from a person's face image, wherein (a) a set of coordinates serving as a feature amount expressing the shape of a head included in the person's face image is input, An estimation algorithm that can output an estimated value of a person's weight, and (b) a set of coordinates of the head included in the person's face image as input, and output correct data of the person's weight, A variable parameter included in the estimation algorithm is automatically learned, and a learning algorithm that can improve the accuracy of the estimated algorithm, and (c) the user of the system adds the facial photograph and the weight at the time of taking the photograph to the system. A machine learning system including a correct data collection support application that can be uploaded. The machine learning of claim 1, wherein the estimation algorithm includes one or more of linear regression, Boltzmann machine, neural network, support vector machine, statistical estimation using Bayesian network, reinforcement learning, deep learning. system. The machine learning system according to claim 1, wherein the learning algorithm includes one or more of linear regression, neural network, support vector machine, statistical estimation using Bayesian network hooks, reinforcement learning, and deep learning. The correct data collection support application for the previous term is a native application or browser application that runs on smartphones and personal computers. The correction processing for improving the machine learning efficiency according to claim 2 and claim 3 for a set of coordinates serving as a feature amount expressing the shape of the head, wherein (a) the accuracy does not depend on the image size (B) Angle correction so that accuracy does not depend on the rotation angle of the face with respect to the vertical or horizontal axis of the image, (c) Position vector of the center coordinate of the face with respect to the center coordinate of the image Correction processing including one or a plurality of parallel movement corrections so as to prevent the accuracy from depending on accuracy. The normalization correction includes a correction method for normalizing the entire image using the distance between the external eye corner points on both sides or the internal eye corner points on the image side obtained by image processing of the head image as a unit of distance. Normalization correction. In the first-stage angle correction, the entire image is rotated so as to increase or decrease the inclination of the straight line formed by connecting the external eye corner points on both sides or the internal eye corner points on both sides obtained by image processing of the head image. Angle correction including method. In the previous translational correction, the coordinates of the midpoint of the straight line formed by connecting the external eye corner points on both sides or the internal eye corner points on both sides obtained by image processing of the head image, or the coordinates of the nose root point Alternatively, parallel correction including a method of performing a parallel movement process on the entire image with one of the coordinates of the interbrow point as the origin.

Images