KR102411882B1 - Untact physical fitness measurement system using images - Google Patents

Abstract

The non-face-to-face physical fitness measurement system using an image includes a portable electronic device and a server. A physical fitness measurement target is photographed with the portable electronic device, and a physical fitness measurement result is calculated using the photographed image. In this process, a deep learning algorithm is used to improve the accuracy of the physical fitness measurement result. The non-face-to-face physical fitness measurement system using the image provides a plurality of fitness measurement modes. The physical fitness measurement modes include a walking in place measurement mode, a rising and sitting measurement mode, and a walking and returning mode.

Description

Non-face-to-face physical fitness measurement system using images

The present invention relates to a non-face-to-face physical fitness measurement system, and more particularly, to a system for measuring physical fitness using a video image captured by a portable electronic device.

In general, physical fitness and health have become essential requirements for a happy life, and a lot of time and effort are devoted to this. In particular, for the elderly in an aging society, physical strength and health are becoming the most important requirements.

In addition, through advanced electronic information technology and communication technology, a technology for conveniently managing physical strength or health is being developed.

However, in the conventional physical fitness measurement method, since the measurement target had to receive the help of a measurement supervisor, there was a problem in that it was difficult for the measurement target to measure the physical strength alone.

In addition, there is a problem that physical fitness can be measured only when the person to be measured and the measurement supervisor meet even in a situation where people are reluctant to meet due to social issues such as infectious diseases.

[Related technical literature]

Body-tailored exercise analysis system (Patent Registration No. 10-2096476)

An object of the present invention is to provide a non-face-to-face physical fitness measurement system that can accurately measure one's own physical fitness without the help of others.

The non-face-to-face physical fitness measurement system using an image according to an embodiment of the present invention may include a portable electronic device including a camera module and a display panel, and a server communicating with the portable electronic device.

The non-face-to-face physical fitness measurement system using the image includes: photographing a person through the camera module; generating an image corresponding to the photographed person on the display panel; extracting a plurality of coordinate information corresponding to a plurality of body parts of the person from the image; detecting whether the person raises his or her left hand or right hand through left wrist coordinate information or right wrist coordinate information among the plurality of coordinate information; detecting whether the person lowered the right hand or the left hand held by the person through the left wrist coordinate information or the right wrist coordinate information; When detecting that the person lowers the right hand or left hand held by the person, right hip coordinate information, left hip coordinate information, right shoulder coordinate information, and left accumulating shoulder coordinate information and storing n pieces (where n is a natural number greater than or equal to 1) for each; calculating a relative ratio between a display area of the display panel and the image using the accumulated coordinate information; A third value is obtained by subtracting a second value obtained by adding y coordinate values among the accumulated right shoulder coordinates from a first value obtained by adding y coordinate values among the accumulated right hip coordinate information, and the accumulated left hip coordinate information A sixth value is obtained by subtracting a fifth value obtained by adding y coordinate values among the accumulated left shoulder coordinates from a fourth value obtained by adding the y coordinate values, and an average value of the third value and the sixth value is obtained, and the average value is obtained. defining an absolute value of a value divided by n as a first noise reference value; accumulating right ankle coordinate information, left ankle coordinate information, right knee coordinate information, and left knee coordinate information among the plurality of coordinate information and storing m pieces (where m is a natural number greater than or equal to 1); applying a Savitzky Golay filter to right ankle coordinate information and left ankle coordinate information among the m pieces of accumulated coordinate information; A seventh value is obtained by subtracting the y coordinate value of the left ankle coordinate information to which the Savitzky Golay filter is applied from the y coordinate value of the right ankle coordinate information to which the Savitzky Golay filter is applied, and the y coordinate of the right knee coordinate information obtaining an eighth value by subtracting the y-coordinate value of the left knee coordinate information from the value, and determining whether a ninth value obtained by adding the seventh value and the eighth value is negative or positive; and when the ninth value is changed from a negative number to a positive number and then from a positive number to a negative number again, determining that the person has performed one gait operation may be performed (included).

In the non-face-to-face physical fitness measurement system using an image according to an embodiment of the present invention, the y-coordinate value change amount of the right ankle coordinate information to which the Savitzky Golay filter is applied and the y-coordinate value change amount of the left ankle coordinate information are the first When it is less than 1 noise reference value, the step of determining that the person has not performed a gait motion may be further performed (included).

In an embodiment of the present invention, information processed through the portable electronic device is transmitted to the server, and the server may learn from the received information using a deep learning algorithm.

In one embodiment of the present invention, the deep learning algorithm may be a convolutional neural network (Convolutional Neural Network) method.

In an embodiment of the present invention, data for a physical fitness measurement criterion is stored in the server, and the server compares the information received from the portable electronic device with the data to calculate a physical fitness measurement result, and the calculated physical fitness measurement A result may be transmitted to the portable electronic device.

When the non-face-to-face physical fitness measurement system using an image according to an embodiment of the present invention detects that the person lowers the right hand or the left hand, among the plurality of coordinate information for a predetermined second time accumulating and storing a plurality of first average coordinate information and nose coordinate information of the left shoulder coordinate information and the right shoulder coordinate information; defining a distance between the average value of the accumulated first average coordinate information and the average value of the accumulated nose coordinate information as a second noise reference value; defining the average value of the accumulated nose coordinate information as a reference coordinate for determination; accumulating and storing a plurality of pieces of second average coordinate information of left shoulder coordinate information and right shoulder coordinate information among the plurality of pieces of coordinate information after the second predetermined time has elapsed; applying a Savitzky-Golay filter to the accumulated second average coordinate information; obtaining a difference between the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied and the y-coordinate value of the determination reference coordinate, and determining whether the difference is negative or positive; and when the car is changed from a positive number to a negative number again after the car is changed from a negative number to a positive number, determining that the person has performed a sitting motion after getting up once may be further performed (included).

In the non-face-to-face physical fitness measurement system using an image according to an embodiment of the present invention, when the amount of change in the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied is smaller than the second noise reference value, the person A step of determining that the motion of standing up and sitting is not performed may be further performed (included).

When the non-face-to-face physical fitness measurement system using an image according to an embodiment of the present invention detects that the person lowers the right hand or the left hand, one of the plurality of coordinate information for a predetermined third time Right ankle coordinate information, left ankle coordinate information, any one of the right and left eye coordinate information, the other of the right and left hip coordinate information, right knee coordinate information, and left knee coordinate information are accumulated and stored in plurality step; The relative ratio between the display area of the display panel and the image is calculated using right knee coordinate information, left knee coordinate information, right ankle coordinate information, and left ankle coordinate information among the coordinate information accumulated for the third predetermined time. calculating; The average of the right ankle coordinate information and the average of the left ankle coordinate information among the coordinates accumulated for the third predetermined time are defined as a departure determination reference coordinate group, and the average of the one eye coordinate information and the other hip defining an average of coordinate information as an arrival determination reference coordinate group; After the third predetermined time has elapsed, a plurality of accumulating and storing right ankle coordinate information, left ankle coordinate information, one eye coordinate information, and the other hip coordinate information among the plurality of coordinate information step; applying a Savitzky-Golay filter to the accumulated one eye coordinate information and the other accumulated hip coordinate information after the third predetermined time has elapsed; determining whether the person has started walking by comparing the accumulated right ankle coordinate information and the accumulated left ankle coordinate information with the departure determination reference coordinate group after the third predetermined time has elapsed; and after the third predetermined time has elapsed, the one eye coordinate information to which the Savitzky Golay filter is applied and the other hip coordinate information to which the Savitzky Golay filter is applied are combined with the arrival determination reference coordinates. Comparing with the group, the step of determining whether the person has returned to the original seat may be further performed (included).

The non-face-to-face physical fitness measurement system using an image according to an embodiment of the present invention includes a portable electronic device including a camera module and a display panel, and a server communicating with the portable electronic device, and can provide a plurality of physical fitness measurement modes have. When a first physical fitness measurement mode is selected from among the plurality of physical fitness measurement modes through the portable electronic device, a person is photographed through the camera module, an image corresponding to the photographed person is displayed on the display panel, and the Extracts a plurality of coordinate information corresponding to a plurality of body parts of the person from the image, and the person raises his or her left hand or right hand through left wrist coordinate information or right wrist coordinate information among the plurality of coordinate information In the case of detecting whether the user has raised or lowered the right hand or the left hand, right hip coordinate information, left hip coordinate information, and right shoulder among the plurality of coordinate information for a predetermined first time The coordinate information and the left shoulder coordinate information are accumulated to store n pieces (where n is a natural number greater than or equal to 1) for each, and the relative ratio between the display area of the display panel and the image using the accumulated coordinate information Calculate, obtain a third value by subtracting a second value obtained by adding y coordinate values among the accumulated right shoulder coordinates from a first value obtained by adding y coordinate values of the accumulated right hip coordinate information to obtain a third value, and the accumulated left hip A sixth value is obtained by subtracting a fifth value obtained by adding y coordinate values among the accumulated left shoulder coordinates from a fourth value obtained by adding y coordinate values among the coordinate information, and an average value of the third value and the sixth value is obtained, , define an absolute value of the average value divided by the n as a first noise reference value, and accumulate right ankle coordinate information, left ankle coordinate information, right knee coordinate information, and left knee coordinate information among the plurality of coordinate information to store m pieces (where m is a natural number greater than or equal to 1), and apply a Savitzky Golay filter to the right ankle coordinate information and the left ankle coordinate information among the m accumulated coordinate information, and the Savitzky Obtaining the difference between the y coordinate value of the right ankle coordinate information to which the Golay filter is applied and the y coordinate value of the left ankle coordinate information to which the Savitzky Golay filter is applied, and determining whether the difference is negative or positive, When the car changes from a negative number to a positive number and then from a positive number to a negative number again, it may be determined that the person has performed one gait motion.

In an embodiment of the present invention, when the amount of change in the y-coordinate value of the right ankle coordinate information to which the Savitzky-Golay filter is applied and the amount of change in the y-coordinate value in the left ankle coordinate information are equal to or less than the first noise reference value, the person It can be determined that the gait motion is not performed.

In an embodiment of the present invention, information processed through the portable electronic device is transmitted to the server, and the server may learn from the received information using a deep learning algorithm.

In an embodiment of the present invention, data for a physical fitness measurement criterion is stored in the server, and the server compares the information received from the portable electronic device with the data to calculate a physical fitness measurement result, and the calculated physical fitness measurement A result may be transmitted to the portable electronic device.

In one embodiment of the present invention, when a second physical fitness measurement mode is selected from among the plurality of physical fitness measurement modes through the portable electronic device, when detecting that the person raises and lowers their right hand or left hand , accumulating and storing a plurality of first average coordinate information and nose coordinate information of the left shoulder coordinate information and the right shoulder coordinate information among the plurality of coordinate information for a predetermined second time, and the accumulated first average coordinate information A distance between an average value and the average value of the accumulated nose coordinate information is defined as a second noise reference value, and the average value of the accumulated nose coordinate information is defined as a judgment reference coordinate, and after the second predetermined time elapses, the Accumulating and storing a plurality of pieces of second average coordinate information of left shoulder coordinate information and right shoulder coordinate information among a plurality of coordinate information, applying a Savitzky Golay filter to the accumulated second average coordinate information, and Obtaining the difference between the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied and the y-coordinate value of the judgment reference coordinate, determining whether the difference is negative or positive, and after the difference is changed from negative to positive, again When the number is changed from a positive number to a negative number, it may be determined that the person has performed a sitting motion after standing up once.

In an embodiment of the present invention, when the amount of change in the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied is smaller than the second noise reference value, it is determined that the person does not stand up and sit down. can judge

In an embodiment of the present invention, when a third physical fitness measurement mode is selected from among the plurality of physical fitness measurement modes through the portable electronic device, when detecting that the person raises and lowers their right hand or left hand , right ankle coordinate information, left ankle coordinate information, any one of the right and left eye coordinate information, the other of the right and left hip coordinate information, and right knee coordinate information among the plurality of coordinate information for a third predetermined time Information and left knee coordinate information are accumulated and stored in plurality, and right knee coordinate information, left knee coordinate information, right ankle coordinate information, and left ankle coordinate information are used among the coordinate information accumulated for the third predetermined time. to calculate the relative ratio between the display area of the display panel and the image, and the average of the right ankle coordinate information and the average of the left ankle coordinate information among the coordinates accumulated for the third predetermined time is used as the departure determination reference coordinate group and defining the average of the one eye coordinate information and the average of the hip coordinate information of the other side as an arrival determination reference coordinate group, and after the third predetermined time elapses, the right of the plurality of coordinate information The ankle coordinate information, the left ankle coordinate information, the one eye coordinate information, and the other hip coordinate information are accumulated and stored in plurality, and the accumulated one eye coordinate information and the accumulated other coordinate information are stored. The Savitzky Golay filter is applied to the hip coordinate information, and the accumulated right ankle coordinate information and the accumulated left ankle coordinate information are compared with the departure determination reference coordinate group to determine whether the person started walking and comparing the one eye coordinate information and the accumulated hip coordinate information of the other side to which the Savitzky Golay filter is applied with the arrival determination reference coordinate group to determine whether the person has returned to the original seat have.

According to an embodiment of the present invention, a person to be measured for physical fitness can accurately measure his/her own physical strength without the help of another person.

Accordingly, the physical fitness measurement target may increase the frequency of measuring the physical strength. In addition, the subject of physical fitness measurement can safely measure his/her own physical strength in a non-face-to-face manner even in times of epidemics.

1 illustrates a non-face-to-face physical fitness measurement system according to an embodiment of the present invention.
Each of FIGS. 2A to 2D exemplarily shows that a physical fitness measurement is performed using a portable electronic device.
Each of FIGS. 3A to 3D exemplarily shows that a physical fitness measurement is performed using a portable electronic device.
Each of FIGS. 4A to 4D exemplarily shows that a physical fitness measurement is performed using a portable electronic device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the drawings, proportions and dimensions of components are exaggerated for effective description of technical content. “and/or” includes any combination of one or more that the associated configurations may define.

A term such as “comprises” is intended to designate that a feature, number, step, action, component, part, or combination thereof described in the specification exists, and includes one or more other features or number, step, action, configuration It should be understood that the possibility of the presence or addition of elements, parts or combinations thereof is not precluded in advance.

1 illustrates a non-face-to-face fitness measurement system (FMS) according to an embodiment of the present invention.

The non-face-to-face fitness measurement system (FMS) may include a portable electronic device (USS), a server (SV), and a personal computer (CD).

The portable electronic device (USS) may include a camera module, a memory device, a control device, a communication module, a display panel, and a touch sensing module. The portable electronic device (USS) may be, for example, a smartphone, a tablet PC, a laptop, a smart watch, or smart glasses. In the present specification, it has been exemplified that the portable electronic device (USS) is a smartphone.

The server SV may communicate with the portable electronic device USS in a wireless or wired manner. The server SV may store data for deriving a physical fitness measurement result using information received from the portable electronic device USS.

The server (SV) may use deep learning or machine learning to improve the accuracy of image analysis taken through the portable electronic device (USS), and based on this, the accuracy of calculating the physical fitness measurement result may be improved. For example, a convolutional neural network (CNN) algorithm may be used, but is not limited thereto.

The server SV may calculate a physical fitness measurement result and transmit it to the portable electronic device USS.

The personal computer CD may communicate with the server SV in a wireless or wired manner. The personal computer CD may control the server SV.

In another embodiment of the present invention, the non-face-to-face physical fitness measurement system (FMS) may be understood as a concept including a portable electronic device (USS) and a server (SV), except for a personal computer (CD).

A non-face-to-face fitness measurement system (FMS) may provide a plurality of fitness measurement modes. For example, the physical fitness measurement modes may include a walking in place measurement mode, a standing up and sitting measurement mode, and a walking and returning mode.

Each of FIGS. 2A to 2D exemplarily shows that a physical fitness measurement is performed using a portable electronic device (USS). Each of FIGS. 2A to 2D exemplarily shows that a user's motion is photographed using a portable electronic device (USS) after a user selects a walking in place measurement mode among physical fitness measurement modes.

The portable electronic device USS may photograph a person and display an image IM corresponding to the photographed person on the display area DA of the display panel. The display area DA may be parallel to a plane defined by the first direction DR1 and the second direction DR2 . The first direction DR1 may be parallel to the X-axis direction, and the second direction DR2 may be parallel to the Y-axis direction. Accordingly, the position of each of the pixels in the display area DA may be expressed as coordinates of the form (x, y).

A portable electronic device (USS) analyzes an image (IM) to obtain a plurality of coordinate information (RY, LY, CN, RS, LS, RE, LE, RW, LW, RH, LH, RK, LK, RA, LA) and extracted coordinate information (RY, LY, CN, RS, LS, RE, LE, RW, LW, RH, LH, RK, LK, RA, LA) movement can be grasped.

The coordinate information (RY, LY, CN, RS, LS, RE, LE, RW, LW, RH, LH, RK, LK, RA, LA) is the right eye coordinate information (RY), the left eye coordinate information (LY) , nose coordinate information (CN), right shoulder coordinate information (RS), left shoulder coordinate information (LS), right elbow coordinate information (RE), left elbow coordinate information (LE), right wrist coordinate information (RW), left wrist Coordinate information (LW), right hip coordinate information (RH), left hip coordinate information (LH), right knee coordinate information (RK), left knee coordinate information (LK), right ankle coordinate information (RA), and left ankle coordinate Information LA may be included.

Referring to FIG. 2A , it is possible to detect whether the person raises the left hand or the right hand through the left wrist coordinate information LW or the right wrist coordinate information RW.

Referring to FIG. 2B , it is possible to detect whether the person lowers the left hand or the right hand through the left wrist coordinate information LW or the right wrist coordinate information RW.

When raising the hand, the left wrist coordinate information (LW) or The coordinate change of the right wrist coordinate information RW is the largest. Therefore, by using the left wrist coordinate information LW or the right wrist coordinate information RW, it is possible to accurately determine whether the raising and lowering of the hand is performed.

When it is determined that the person has raised and lowered the right hand or the left hand held by the person, the coordinate information (RY, LY, CN, RS, LS, RE, Among LE, RW, LW, RH, LH, RK, LK, RA, LA), right hip coordinate information (RH), left hip coordinate information (LH), right shoulder coordinate information (RS), and left shoulder coordinate information (LS) ) and store n (where n is a natural number greater than or equal to 1). In this case, the accumulated information may be stored in a memory of the portable electronic device USS.

The portable electronic device (USS) uses the right hip coordinate information (RH), the left hip coordinate information (LH), the right shoulder coordinate information (RS), and the left shoulder coordinate information (LS) accumulated for a predetermined time in the display area A relative ratio between (DA) and an image (IM) corresponding to a person may be calculated.

The distance between the portable electronic device (USS) and the person (physical test subject), the height of the person (physical test subject), the relative position between the portable electronic device (USS) and the person (physical test subject), the portable electronic device (USS) The ratio of the image IM in the display area DA may vary depending on the performance of the camera module mounted on the . In this way, in consideration of the relative ratio of the image IM in the display area DA, the coordinate information RY, LY, CN, RS, LS, RE, LE, RW, LW, RH, LH, RK, If the movement of LK, RA, LA) is detected, more accurate measurement is possible.

The right hip coordinate information (RH), the left hip coordinate information (LH), the right shoulder coordinate information (RS), and the left shoulder coordinate information (LS) correspond to a human torso and are large enough to facilitate comparison of relative proportions. In addition, since the right hip coordinate information (RH), the left hip coordinate information (LH), the right shoulder coordinate information (RS), and the left shoulder coordinate information (LS) hardly change the movement in the process of performing the walking motion in place. , it is appropriate to use as reference coordinates for measuring relative proportions in the walking in place measurement mode.

In addition, the portable electronic device (USS) uses the right hip coordinate information (RH), the left hip coordinate information (LH), the right shoulder coordinate information (RS), and the left shoulder coordinate information (LS) accumulated for a predetermined time. A first noise reference value may be defined. The reason why the first noise reference value is defined is to prevent an error from being counted as if an accurate walking in place was performed even with a small movement.

A first value is obtained by adding the y coordinate values among the accumulated n pieces of right hip coordinate information (RH), and a second value is obtained by adding the y coordinate values of the accumulated n pieces of right shoulder coordinate information (RS), and a first value is obtained. The third value can be obtained by subtracting the second value from the value. A fourth value is obtained by adding the y coordinate values of the accumulated n pieces of left hip coordinate information LH, and a fifth value is obtained by adding the y coordinate values of the accumulated n pieces of left shoulder coordinate information LS, and a fourth value is obtained. The sixth value can be obtained by subtracting the fifth value from the value. An average value of the third value and the sixth value may be obtained, and an absolute value of a value obtained by dividing the average value by n may be defined as the first noise reference value.

After the person raises and lowers the right hand or the left hand held by the person, the movement of the person may be measured in earnest after a predetermined time (eg, about 3 seconds) elapses.

Specifically, in order to determine whether to perform a walking motion in place, right ankle coordinate information (RA), left ankle coordinate information (LA), right knee coordinate information (RK), and left knee coordinate information (LK) are accumulated to m It can store a number (where m is a natural number greater than or equal to 1).

A Savitzky-Golay Filter may be applied to the right ankle coordinate information RA and the left ankle coordinate information LA among the accumulated m pieces of coordinate information. A Savitzky-Golay filter is a digital filter that can be used to remove noise in the process of observing wave-shaped data.

When the Savitzky-Golay Filter is applied to all of the right ankle coordinate information (RA), the left ankle coordinate information (LA), the right knee coordinate information (RK), and the left knee coordinate information (LK), Face-to-face fitness systems (FMS) may slow down. Accordingly, considering that the right ankle coordinate information RA and the left ankle coordinate information LA generate more noise than the right knee coordinate information RK and the left knee coordinate information LK, It is preferable to apply the Savitzky-Golay Filter only to the right ankle coordinate information RA and the left ankle coordinate information LA.

When the amount of change in the y-coordinate of the right ankle coordinate information (RA) and the amount of change in the y-coordinate in the left ankle coordinate information (LA) to which the Savitzky Golay filter is applied are smaller than the first noise reference value, the subject of physical fitness measurement performs the gait motion. may be judged not to have been performed.

2c and 2d, when the amount of change in the y-coordinate of the right ankle coordinate information (RA) to which the Savitzky Golay filter is applied and the amount of change in the y-coordinate in the left ankle coordinate information (LA) are equal to or greater than the first noise reference value, The seventh value is obtained by subtracting the y coordinate value of the left ankle coordinate information (LA) from the y coordinate value of the right ankle coordinate information (RA) to which the Savitzky Golay filter is applied, and the y coordinate value of the right knee coordinate information (RK) The eighth value can be obtained by subtracting the y-coordinate value of the left knee coordinate information from . Thereafter, it may be determined whether the ninth value obtained by adding the seventh value and the eighth value is a negative number or a positive number.

If the ninth value calculated as above is changed from negative (positive) to positive (negative) and then changes from positive (negative) to negative (positive), it is judged that the subject of physical fitness performed one gait motion at this time. can do.

In this way, by considering not only the relative position between the right ankle coordinate information RA and the left ankle coordinate information LA, but also the relative position between the right knee coordinate information RK and the left knee coordinate information LK, the gait It is possible to accurately determine whether an operation is performed or not.

Also, in the display area DA, an image TM corresponding to the remaining time in the process of measuring the walking motion in place and an image CT corresponding to the number of times the motion is performed may be displayed. For example, the image TM corresponding to the remaining time may be displayed in such a way that the remaining time gradually decreases from 120 seconds to 0 seconds. The image CT corresponding to the number of times the operation is performed may be displayed in such a way that the number increases by 0.5 whenever one foot goes up, and increases by 1 when both feet go up and then go down.

Each of FIGS. 3A to 3D exemplarily shows that a physical fitness measurement is performed using a portable electronic device (USS). Each of FIGS. 3A to 3D exemplarily shows that a user's motion is photographed using a portable electronic device (USS) after a user selects a rising and sitting measuring mode from among the physical fitness measurement modes.

As shown in FIGS. 3A and 3B , when it is determined that the person has raised and lowered the right hand or the left hand held by the person, the coordinate information for a predetermined time (eg, about 3 seconds) ( Average coordinate information of right shoulder coordinate information (RS) and left shoulder coordinate information (LS) among RY, LY, CN, RS, LS, RE, LE, RW, LW, RH, LH, RK, LK, RA, LA) (hereinafter, the first average coordinate information) and the nose coordinate information (CN) may be accumulated and stored in plurality. In this case, the accumulated information may be stored in a memory of the portable electronic device USS.

The accumulated nose coordinate information CN may be defined as a judgment reference coordinate.

The portable electronic device USS may define a second noise reference value using the first average coordinate information and the nose coordinate information CN accumulated for a predetermined time. The reason why the second noise reference value is defined is to prevent a counting error as if a motion of getting up and sitting from a chair was performed even with a small movement.

A distance between the average value of the accumulated first average coordinate information and the average value of the accumulated nose coordinate information CN may be defined as the second noise reference value.

After the person raises and lowers the right hand or the left hand held by the person, the movement of the person may be measured in earnest after a predetermined time (eg, about 3 seconds) elapses.

Specifically, in order to determine whether to perform a sitting motion after rising from a chair, the average coordinate information (hereinafter, second average coordinate information) of the left shoulder coordinate information (LS) and the right shoulder coordinate information (RS) is accumulated and a plurality of can be saved

A Savitzky-Golay filter may be applied to the accumulated second average coordinate information.

When the amount of change in the y-coordinate of the second average coordinate information to which the Savitzky Golay filter is applied is smaller than the second noise reference value, it may be determined that the subject of the physical fitness measurement does not stand up or sit down.

When the amount of change of the y-coordinate of the second average coordinate information to which the Savitzky-Golay filter is applied is equal to or greater than the second noise reference value, the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied and the y-coordinate of the judgment reference coordinate You can find the difference between values and determine whether the difference is negative or positive.

After the difference between the y-coordinate value of the second average coordinate information and the y-coordinate value of the judgment reference coordinate changes from a negative number (positive number) to a positive number (negative number), when it changes from a positive number (negative number) to a negative number (positive number) again, the subject of physical fitness measurement It can be determined that the sitting motion was performed after getting up once.

In this way, by setting the nose coordinate information (CN) in the sitting state as the judgment reference coordinates, and comparing the average value of the moving right shoulder coordinate information (RS) and the left shoulder coordinate information (LS) with the judgment reference coordinates, physical fitness measurement It is possible to accurately judge whether the subject performed the sitting motion or not.

In addition, in the display area DA, an image TM corresponding to the remaining time in the process of measuring the rising and sitting motion and an image CT corresponding to the number of times the motion is performed may be displayed. For example, the image TM corresponding to the remaining time may be displayed in such a way that the remaining time gradually decreases from 30 seconds to 0 seconds. The image CT corresponding to the number of times the motion is performed may be displayed in such a way that the number increases by 0.5 every time one gets up or sits down, and increases by one when all the motions of getting up and sitting are performed.

Other descriptions are substantially the same as those described with reference to FIGS. 2A to 2D, and thus will be omitted.

Each of FIGS. 4A to 4D exemplarily shows that a physical fitness measurement is performed using a portable electronic device (USS). Each of FIGS. 4A to 4D exemplarily shows that a user's motion is photographed using a portable electronic device (USS) after the user selects a return mode after walking among the physical fitness measurement modes.

As shown in FIGS. 4A and 4B , when it is determined that the person has raised and lowered the right hand or the left hand held by the person, the coordinate information (for example, about 3 seconds) for a predetermined time (for example, about 3 seconds) Among RY, LY, CN, RS, LS, RE, LE, RW, LW, RH, LH, RK, LK, RA, LA), right ankle coordinate information (RA), left ankle coordinate information (LA), right and left One of the eye coordinate information (RY or LY), the hip coordinate information (RH or LH) of the other of the right and left, the right knee coordinate information (RK), and the left knee coordinate information (LK) can be saved For example, when the right eye coordinate information RY is accumulated and stored, the opposite left hip coordinate information LH may be accumulated and stored. When the left eye coordinate information LY is accumulated and stored, the opposite right hip coordinate information RH may be accumulated and stored. In this case, the accumulated information may be stored in a memory of the portable electronic device USS.

The portable electronic device (USS) collects the right knee coordinate information (RK), the left knee coordinate information (LK), the right ankle coordinate information (RA), and the left ankle coordinate information (LA) among the coordinate information accumulated for a predetermined time. A relative ratio between the display area DA and the image IM corresponding to the person may be calculated using the DA.

An average of the right ankle coordinate information RA and an average of the left ankle coordinate information LA among the coordinates accumulated for a predetermined time may be defined as a departure determination reference coordinate group. Then, the average of one eye coordinate information (eg, the right eye coordinate information (RY)) and the average of the other hip coordinate information (eg, the left hip coordinate information (LH)) are determined to arrive It can be defined as a reference coordinate group. Since there are two coordinates instead of one, it is defined in the form of a group.

As described above, by selecting the eye coordinate information and the hip coordinate information on different sides from the right and the left, it is possible to reduce errors in erroneous judgment of human motion and improve accuracy.

After the person raises and lowers the right hand or the left hand held by the person, the movement of the person may be measured in earnest after a predetermined time (eg, about 3 seconds) elapses.

Specifically, the right ankle coordinate information (RA), the left ankle coordinate information (LA), one eye coordinate information (RY or LY), and the other hip coordinate information (LH or RH) may be accumulated and stored in plurality.

A Savitzky-Golay Filter may be applied to the accumulated one eye coordinate information (RY or LY) and the other hip coordinate information (LH or RH).

By comparing the accumulated right ankle coordinate information RA and the accumulated left ankle coordinate information LA with the departure determination reference coordinates, it may be determined whether the physical fitness subject has started walking.

By comparing the coordinate information of one eye (RY or LY) and the hip coordinate information (LH or RH) of one side to which the Savitzky Golay filter is applied with the reference coordinates for determining arrival, it is determined whether the person to be measured has returned to the original position can do.

In this way, when determining whether the subject for physical fitness measurement has departed, the right ankle coordinate information (RA) and the left ankle coordinate information (LA) are used, and when determining whether the target has arrived, one eye coordinate information (RY or LY) and the other By using the hip coordinate information (LH or RH), it is possible to accurately determine whether the motion of walking and returning is performed.

Also, in the display area DA, an image EC corresponding to a condition for terminating the measurement and an image OT corresponding to an elapsed time may be displayed in the process of measuring the rising and sitting motion.

Other descriptions are substantially the same as those described with reference to FIGS. 2A to 2D, and thus will be omitted.

Although described with reference to embodiments, those skilled in the art can understand that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention described in the claims below. There will be. In addition, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, and all technical ideas within the scope of the following claims and their equivalents should be construed as being included in the scope of the present invention. .

FMS: Non-face-to-face fitness measurement system
USS: Portable Electronics
SV: Server
CD: personal computer

Claims (15)

In the non-face-to-face physical fitness measurement system using an image comprising a portable electronic device including a camera module and a display panel and a server communicating with the portable electronic device,
The non-face-to-face physical fitness measurement system using the image,
photographing a person through the camera module;
generating an image corresponding to the photographed person on the display panel;
extracting a plurality of coordinate information corresponding to a plurality of body parts of the person from the image;
detecting whether the person raises his or her left hand or right hand through left wrist coordinate information or right wrist coordinate information among the plurality of coordinate information;
detecting whether the person lowered the right hand or the left hand held by the person through the left wrist coordinate information or the right wrist coordinate information;
When detecting that the person lowers the right hand or the left hand held by the person, right hip coordinate information, left hip coordinate information, right shoulder coordinate information, and left side of the plurality of coordinate information for a predetermined first time accumulating shoulder coordinate information and storing n pieces (where n is a natural number greater than or equal to 1) for each;
calculating a relative ratio between a display area of the display panel and the image using the accumulated coordinate information;
A third value is obtained by subtracting a second value obtained by adding y coordinate values among the accumulated right shoulder coordinates from a first value obtained by adding y coordinate values among the accumulated right hip coordinate information, and the accumulated left hip coordinate information A sixth value is obtained by subtracting a fifth value obtained by adding y coordinate values among the accumulated left shoulder coordinates from a fourth value obtained by adding the y coordinate values, and an average value of the third value and the sixth value is obtained, and the average value is obtained. defining an absolute value of a value divided by n as a first noise reference value;
storing m pieces (where m is a natural number greater than or equal to 1) by accumulating right ankle coordinate information, left ankle coordinate information, right knee coordinate information, and left knee coordinate information among the plurality of coordinate information;
applying a Savitzky Golay filter to right ankle coordinate information and left ankle coordinate information among the m pieces of accumulated coordinate information;
A seventh value is obtained by subtracting the y coordinate value of the left ankle coordinate information to which the Savitzky Golay filter is applied from the y coordinate value of the right ankle coordinate information to which the Savitzky Golay filter is applied, and the y coordinate of the right knee coordinate information obtaining an eighth value by subtracting the y-coordinate value of the left knee coordinate information from the value, and determining whether a ninth value obtained by adding the seventh value and the eighth value is negative or positive; and
After the ninth value is changed from negative to positive and then changed from positive to negative again, the image of performing a non-face-to-face physical fitness measurement method using an image comprising the step of determining that the person has performed one gait operation A non-face-to-face physical fitness measurement system. According to claim 1,
The non-face-to-face physical fitness measurement method using the image,
When the amount of change in the y-coordinate value of the right ankle coordinate information to which the Savitzky Golay filter is applied and the amount of change in the y-coordinate value in the left ankle coordinate information are smaller than the first noise reference value, the person does not perform a gait motion Non-face-to-face physical fitness measurement system using an image further comprising the step of determining that the 3. The method of claim 2,
The information processed through the portable electronic device is transmitted to the server, and the server is a non-face-to-face physical fitness measurement system using an image that learns from the received information using a deep learning algorithm. 4. The method of claim 3,
The deep learning algorithm is a convolutional neural network (Convolutional Neural Network) non-face-to-face fitness measurement system using an image. 3. The method of claim 2,
Data for the physical fitness measurement standard is stored in the server, the server compares the information received from the portable electronic device with the data to calculate a physical fitness measurement result, and transmits the calculated physical fitness measurement result to the portable electronic device A non-face-to-face physical fitness measurement system using the image of 3. The method of claim 2,
The non-face-to-face physical fitness measurement method using the image,
When detecting that the person lowers the right hand or the left hand held by the person, the first average coordinate information and the nose of the left shoulder coordinate information and the right shoulder coordinate information among the plurality of coordinate information for a predetermined second time accumulating and storing a plurality of coordinate information;
defining a distance between the average value of the accumulated first average coordinate information and the average value of the accumulated nose coordinate information as a second noise reference value;
defining the average value of the accumulated nose coordinate information as a reference coordinate for determination;
accumulating and storing a plurality of pieces of second average coordinate information of left shoulder coordinate information and right shoulder coordinate information among the plurality of pieces of coordinate information after the second predetermined time has elapsed;
applying a Savitzky-Golay filter to the accumulated second average coordinate information;
obtaining a difference between the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied and the y-coordinate value of the determination reference coordinate, and determining whether the difference is negative or positive; and
The non-face-to-face physical fitness measurement system using an image further comprising the step of determining that the person has performed a sitting motion after getting up once when the car changes from a positive number to a negative number after the car is changed from a negative number to a positive number. 7. The method of claim 6,
The non-face-to-face physical fitness measurement method using the image,
When the amount of change in the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied is smaller than the second noise reference value, determining that the person does not stand up and sit down Image further comprising: A non-face-to-face physical fitness measurement system using 3. The method of claim 2,
The non-face-to-face physical fitness measurement method using the image,
When it is detected that the person lowers the right hand or left hand held by the person, any one of right ankle coordinate information, left ankle coordinate information, and right and left from among the plurality of coordinate information for a predetermined third time period accumulating and storing a plurality of eye coordinate information, the other of the right and left hip coordinate information, right knee coordinate information, and left knee coordinate information;
The relative ratio between the display area of the display panel and the image is calculated using right knee coordinate information, left knee coordinate information, right ankle coordinate information, and left ankle coordinate information among the coordinate information accumulated for the third predetermined time. calculating;
The average of the right ankle coordinate information and the average of the left ankle coordinate information among the coordinates accumulated for the predetermined third time are defined as a departure determination reference coordinate group, and the average of the one eye coordinate information and the other hip defining an average of coordinate information as an arrival determination reference coordinate group;
After the third predetermined time has elapsed, a plurality of accumulating and storing right ankle coordinate information, left ankle coordinate information, one eye coordinate information, and the other hip coordinate information among the plurality of coordinate information step;
applying a Savitzky-Golay filter to the accumulated one eye coordinate information and the other accumulated hip coordinate information after the third predetermined time has elapsed;
determining whether the person has started walking by comparing the accumulated right ankle coordinate information and the accumulated left ankle coordinate information with the departure determination reference coordinate group after the third predetermined time has elapsed; and
After the third predetermined time has elapsed, the one eye coordinate information to which the Savitzky Golay filter is applied and the other hip coordinate information to which the Savitzky Golay filter is applied are added to the arrival determination reference coordinate group A non-face-to-face physical fitness measurement system using an image further comprising the step of determining whether the person has returned to the original seat by comparing with the . A portable electronic device including a camera module and a display panel, and a server that communicates with the portable electronic device, and provides a plurality of physical fitness measurement modes,
When the first physical fitness measurement mode is selected from among the plurality of physical fitness measurement modes through the portable electronic device,
photographing a person through the camera module, displaying an image corresponding to the photographed person on the display panel, and extracting a plurality of coordinate information corresponding to a plurality of body parts of the person from the image;
Detects whether the person raises or lowers the left hand or the right hand through the left wrist coordinate information or the right wrist coordinate information among the plurality of coordinate information,
When detecting that the person raises and lowers the right hand or the left hand, right hip coordinate information, left hip coordinate information, right shoulder coordinate information, and left shoulder from among the plurality of coordinate information for a predetermined first time By accumulating coordinate information, n pieces (where n is a natural number greater than or equal to 1) are stored for each,
calculating a relative ratio between the display area of the display panel and the image using the accumulated coordinate information;
A third value is obtained by subtracting a second value obtained by adding y coordinate values among the accumulated right shoulder coordinates from a first value obtained by adding y coordinate values among the accumulated right hip coordinate information, and the accumulated left hip coordinate information A sixth value is obtained by subtracting a fifth value obtained by adding y coordinate values among the accumulated left shoulder coordinates from a fourth value obtained by adding the y coordinate values, and an average value of the third value and the sixth value is obtained, and the average value is obtained. An absolute value of a value divided by n is defined as a first noise reference value,
Among the plurality of coordinate information, right ankle coordinate information, left ankle coordinate information, right knee coordinate information, and left knee coordinate information are accumulated to store m pieces (where m is a natural number greater than or equal to 1),
applying the Savitzky Golay filter to the right ankle coordinate information and the left ankle coordinate information among the m pieces of accumulated coordinate information;
Obtaining the difference between the y coordinate value of the right ankle coordinate information to which the Savitzky Golay filter is applied and the y coordinate value of the left ankle coordinate information to which the Savitzky Golay filter is applied, and determining whether the difference is negative or positive,
A non-face-to-face physical fitness measurement system using an image for determining that the person has performed one gait motion when the car changes from a negative number to a positive number and then from a positive number to a negative number again. 10. The method of claim 9,
When the amount of change in the y-coordinate value of the right ankle coordinate information to which the Savitzky-Golay filter is applied and the amount of change in the y-coordinate value in the left ankle coordinate information are less than or equal to the first noise reference value, it is determined that the person has not performed the gait motion. A non-face-to-face physical fitness measurement system using images to judge. 11. The method of claim 10,
The information processed through the portable electronic device is transmitted to the server, and the server is a non-face-to-face physical fitness measurement system using an image that learns from the received information using a deep learning algorithm. 11. The method of claim 10,
Data for the physical fitness measurement standard is stored in the server, the server compares the information received from the portable electronic device with the data to calculate a physical fitness measurement result, and transmits the calculated physical fitness measurement result to the portable electronic device A non-face-to-face physical fitness measurement system using the image of 11. The method of claim 10,
When a second fitness measurement mode is selected from among the plurality of fitness measurement modes through the portable electronic device,
When detecting that the person raises and lowers the right hand or the left hand, the first average coordinate information and the nose of the left shoulder coordinate information and the right shoulder coordinate information among the plurality of coordinate information for a predetermined second time By accumulating and storing coordinate information,
A distance between the average value of the accumulated first average coordinate information and the average value of the accumulated nose coordinate information is defined as a second noise reference value,
The average value of the accumulated nose coordinate information is defined as a reference coordinate for determination,
After the second predetermined time has elapsed, the second average coordinate information of the left shoulder coordinate information and the right shoulder coordinate information among the plurality of coordinate information is accumulated and stored in plurality,
A Savitzky-Golay filter is applied to the accumulated second average coordinate information, and the difference between the y-coordinate value of the second average coordinate information to which the Savitzky-Golay filter is applied and the y-coordinate value of the judgment reference coordinate is calculated to determine whether the difference is negative or positive,
Non-face-to-face physical fitness measurement system using an image for determining that the person has stood up once and then sat down when the car is changed from a negative number to a positive number and then from a positive number to a negative number again. 14. The method of claim 13,
When the amount of change in the y-coordinate value of the second average coordinate information to which the Savitzky Golay filter is applied is smaller than the second noise reference value, non-face-to-face physical fitness using an image in which it is determined that the person does not stand up and sit down measuring system. 11. The method of claim 10,
When a third physical fitness measurement mode is selected from among the plurality of physical fitness measurement modes through the portable electronic device,
When it is detected that the person raises and lowers the right hand or the left hand, any one of right ankle coordinate information, left ankle coordinate information, and right and left among the plurality of coordinate information for a predetermined third time period accumulating and storing a plurality of eye coordinate information, hip coordinate information of the other of the right and left, right knee coordinate information, and left knee coordinate information,
The relative ratio between the display area of the display panel and the image is calculated using right knee coordinate information, left knee coordinate information, right ankle coordinate information, and left ankle coordinate information among the coordinate information accumulated for the third predetermined time. calculate,
The average of the right ankle coordinate information and the average of the left ankle coordinate information among the coordinates accumulated for the predetermined third time are defined as a departure determination reference coordinate group, and the average of the one eye coordinate information and the other hip Define the average of the coordinate information as the arrival judgment reference coordinate group,
After the third predetermined time has elapsed, a plurality of pieces of right ankle coordinate information, left ankle coordinate information, one eye coordinate information, and the other hip coordinate information are accumulated and stored among the plurality of coordinate information, ,
applying a Savitzky Golay filter to the accumulated one eye coordinate information and the other accumulated hip coordinate information;
By comparing the accumulated right ankle coordinate information and the accumulated left ankle coordinate information with the departure determination reference coordinate group, it is determined whether the person started walking,
The image is used to determine whether the person has returned to the original seat by comparing the eye coordinate information of one side to which the Savitzky Golay filter is applied and the accumulated hip coordinate information of the other side with the arrival determination reference coordinate group. Non-face-to-face fitness measurement system.

Images