KR102568538B1 - Apparatus and Method for Estimating Human Motion using Mobile Robot - Google Patents

Abstract

An apparatus and method for evaluating human motion using a mobile robot are disclosed. A method for evaluating human motion using a mobile robot according to an embodiment of the present invention includes the steps of identifying a motion motion of the human by analyzing a full-body image of a human captured by a camera mounted on the mobile robot, and a standard posture corresponding to the identified motion motion. Evaluating the posture of the human by comparing the whole body images of the human photographed at at least two or more target positions by the camera of the mobile robot and the movement motion of the human based on the evaluation information of the human posture at each of the at least two or more target positions may include a comprehensive evaluation of

Description

Apparatus and Method for Estimating Human Motion using Mobile Robot}

The described embodiment relates to a technology in which a robot living with a human evaluates a human's exercise motion, such as fitness, yoga, and stretching, based on image analysis.

Homet (home training), which is an exercise to do alone at home while watching videos on TV or a smartphone, is gaining great popularity and is becoming a trend. Recently, commercial products that use artificial intelligence technology to evaluate and correct posture in real time by recognizing the learner's movement posture are emerging one after another. AI coaching services based on these motion evaluations are expected to spread to various areas such as fitness and diet as well as professional sports and medical rehabilitation, and related markets are expected to grow rapidly.

The core technology of the video analysis-based AI coaching service is the posture information extraction technology that identifies the posture the learner is taking. Most conventional technologies detect the position of a human joint and utilize skeleton information composed of the detected joints as posture information of a learner. Therefore, accurate joint position detection is very important, and the accuracy of joint position detection greatly influences the performance of the entire motion evaluation system.

However, existing techniques for evaluating human motion using a camera attached to a TV or smartphone cannot accurately detect all joints important for motion evaluation. Therefore, there is a problem in that reliable evaluation results cannot be provided to the user. That is, there is an optimal camera direction that is easy to detect the position of each joint according to motion, but there is a limit to accurately detecting all joints with a fixed camera position. In addition, even when a specific joint is covered by another part of the body or a tool, it is difficult to accurately detect the location of the joint. Therefore, the conventional techniques are at a level of providing only partial evaluation results by performing comparative evaluation on only a few joints capable of position detection.

In order to overcome the limitations of a single camera, there are attempts to solve the problem by using multiple cameras (2 to 4 cameras). However, other problems arise such as an increase in installation cost and the need to configure a separate system composed of multiple cameras. In addition, although multiple cameras are advantageous in detecting joint positions, since the cameras are fixed, a fundamental problem of not actively coping with an optimal direction for easy detection of each joint still remains.

Korean Patent Publication No. 10-2020-0022788

An object of the disclosed embodiments is to improve evaluation performance of a posture or motion by observing a human motion in an optimal position while a robot changes its position when a human performs an motion.

A method for evaluating human motion using a mobile robot according to an embodiment includes the steps of analyzing a human whole body image captured by a camera mounted on the mobile robot to identify a motion motion of the human, a standard posture corresponding to the identified motion motion, and a mobile robot Evaluating the posture of the human by comparing whole-body images of the human captured at at least two or more target positions by the camera of the camera, and comprehensively evaluating the motion of the human based on the evaluation information of the human posture at each of the at least two or more target positions. steps may be included.

In this case, the step of evaluating the posture of the human may include detecting a joint in the human whole body image and calculating a posture accuracy score for a segment composed of joints having detection accuracy greater than or equal to a reference value.

At this time, the step of evaluating the posture of the human further includes the step of recording the detection accuracy of the joint and the posture accuracy score for the segment in the posture evaluation diagram, wherein the posture evaluation diagram indicates all segments and joints of the human. can

At this time, the step of evaluating the posture of the human is repeatedly evaluated for each segment, and the step of recording in the posture evaluation diagram may record the number of evaluations of the segment in the posture evaluation diagram so as to be distinguished according to color or brightness.

At this time, in the step of evaluating the posture of the human, the target position is set based on the posture profile, but if the posture profile includes a weight and a 2D direction vector for each segment, the next target segment to be observed is selected in the order of weight. A target position, which is an optimal observation position for the selected segment, may be set based on the 2D direction vector of the selected segment.

At this time, in the step of evaluating the posture of the human, a target position is set based on the posture profile, but when the posture profile includes only weights for each segment, the next observation target segment is selected in order of weight, and the optimal observation is performed. A target position, which is a position, is calculated in real time, and the target position may be set to a point where two joints constituting the selected segment are observed, extend vertically from the center of the segment, and have the same distance from each of the two joints.

At this time, in the step of evaluating the posture of the human, when the posture profile is not provided, a target position may be set while moving a point at a predetermined distance from the human by a predetermined angle.

At this time, the step of evaluating the motion of the human includes calculating the segment detection accuracy as the average of the detection accuracies of the joints constituting the segment, and calculating the final posture accuracy score of the segment as the average of the segment detection accuracy score and the segment detection accuracy. and calculating a final motion accuracy based on a final posture accuracy score and a weight for each segment.

An apparatus for evaluating human motion using a mobile robot according to an embodiment includes a memory in which at least one program is recorded and a processor that executes the program, and the program analyzes a human body image captured by a camera mounted on the mobile robot, Identifying the motion of the human; Evaluating the posture of the human by comparing the standard posture corresponding to the identified motion and the whole body images of the human taken at at least two or more target positions by the camera of the mobile robot; and A step of comprehensively evaluating the motion motion of the human based on the posture evaluation information of the human at each of the target positions may be performed.

In this case, the step of evaluating the posture of the human may include detecting a joint in the human whole body image and calculating a posture accuracy score for a segment composed of joints having detection accuracy greater than or equal to a reference value.

At this time, the step of evaluating the posture of the human further includes the step of recording the detection accuracy of the joint and the posture accuracy score for the segment in the posture evaluation diagram, wherein the posture evaluation diagram indicates all segments and joints of the human. can

At this time, the step of evaluating the posture of the human is repeatedly evaluated for each segment, and the step of recording in the posture evaluation diagram may record the number of evaluations of the segment in the posture evaluation diagram so as to be distinguished according to color or brightness.

At this time, in the step of evaluating the posture of the human, the target position is set based on the posture profile, but if the posture profile includes a weight and a 2D direction vector for each segment, the next target segment to be observed is selected in the order of weight. A target position, which is an optimal observation position for the selected segment, may be set based on the 2D direction vector of the selected segment.

At this time, in the step of evaluating the posture of the human, a target position is set based on the posture profile, but when the posture profile includes only weights for each segment, the next observation target segment is selected in order of weight, and the optimal observation is performed. A target position, which is a position, is calculated in real time, and the target position may be set to a point where two joints constituting the selected segment are observed, extend vertically from the center of the segment, and have the same distance from each of the two joints.

At this time, in the step of evaluating the posture of the human, when the posture profile is not provided, a target position may be set while moving a point at a predetermined distance from the human by a predetermined angle.

At this time, the step of evaluating the motion of the human includes calculating the segment detection accuracy as the average of the detection accuracies of the joints constituting the segment, and calculating the final posture accuracy score of the segment as the average of the segment detection accuracy score and the segment detection accuracy. and calculating a final motion accuracy based on a final posture accuracy score and a weight for each segment.

A method for evaluating human motion using a mobile robot according to an embodiment includes the steps of identifying a motion motion of a human by analyzing a whole body image of a human captured by a camera mounted on the mobile robot, and a standard posture and posture profile corresponding to the identified motion motion. Detecting, setting a target position based on the posture profile, moving the mobile robot to the set target position, evaluating the human posture by comparing the human whole body image captured by the camera and the standard posture, and the target position Comprehensive evaluation of the human's exercise motion based on the evaluation information of the human's posture in each case may be performed, but the steps of setting a target position and evaluating the human's posture may be repeatedly performed.

At this time, the step of evaluating the human posture may include detecting joints in the human body image and repeatedly calculating posture accuracy scores for segments composed of joints having detection accuracy equal to or greater than a reference value for a predetermined number of evaluations. there is.

At this time, in the step of setting the target position, the target position is set based on the posture profile, but when the posture profile includes weights and 2D direction vectors for each segment, the next segment to be observed is selected in the order of weight. And, based on the 2D direction vector of the selected segment, a target position, which is an optimal viewing position for the corresponding segment, can be set.

At this time, the step of setting the target position is to set the target position based on the posture profile, but when the posture profile includes only weights for each segment, select the next observation target segment in order of weight, and select the optimal observation position. The target position may be calculated in real time, and the target position may be set to a point where two joints constituting the selected segment are observed, extend vertically from the center of the segment, and have the same distance from each of the two joints.

Depending on the embodiment, the mobile robot moves by itself to an optimal position that is advantageous for detecting the position of a joint when a human performs a specific motion, and observes the motion of the human in various directions, thereby significantly improving the reliability of posture or motion evaluation. There is an advantage to being

1 is a conceptual diagram of a device for evaluating a human posture using a fixed camera.
2 is a schematic conceptual diagram of a human motion evaluation apparatus using a mobile robot according to an embodiment.
3 is a flowchart illustrating a method for evaluating a human motion using a mobile robot according to an embodiment.
4 is a flowchart illustrating a posture evaluation step according to an embodiment.
5 is an exemplary view of a posture evaluation diagram according to an embodiment.
6 is a flowchart illustrating a step of moving a robot position according to an embodiment.
7 is an exemplary view of a posture profile for setting a target position according to an embodiment.
8 is a flowchart illustrating a step of setting a target location according to an embodiment.
9 is an exemplary diagram for explaining a method for searching for an optimal viewing direction of a segment according to an embodiment.
10 is an exemplary view of determining a target position of a mobile robot when a posture profile is not provided according to an embodiment.
11 is a diagram showing a configuration of a computer system according to an embodiment.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Although "first" or "second" is used to describe various elements, these elements are not limited by the above terms. Such terms may only be used to distinguish one component from another. Therefore, the first component mentioned below may also be the second component within the technical spirit of the present invention.

Terms used in this specification are for describing embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" or "comprising" implies that a stated component or step does not preclude the presence or addition of one or more other components or steps.

Unless otherwise defined, all terms used herein may be interpreted as meanings commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

Hereinafter, a human posture evaluation apparatus and method using a mobile robot according to an embodiment will be described in detail with reference to FIGS. 1 to 11 .

1 is a conceptual diagram of a device for evaluating a human posture using a fixed camera.

Referring to FIG. 1 , a conventional device for evaluating a human posture uses a fixed single camera 10 or multiple cameras 21 and 22 attached to a TV or smart phone.

Such a conventional device for evaluating human posture cannot accurately detect all joints important for motion evaluation. That is, there is an optimal camera direction that is easy to detect the position of each joint according to motion, but there is a limit to accurately detecting all joints with a fixed camera position. In addition, even when a specific joint is covered by another part of the body or a tool, it is difficult to accurately detect the location of the joint. Therefore, the reliability of conventional technologies is low to the level of providing only partial evaluation results by performing comparative evaluation on only a few joints capable of position detection.

2 is a schematic conceptual diagram of a human motion evaluation apparatus using a mobile robot according to an embodiment.

Referring to FIG. 2 , the human motion evaluation apparatus 100 using a mobile robot according to an embodiment may be implemented using a mobile robot equipped with a camera.

The human motion evaluation apparatus 100 improves the accuracy of joint detection and ultimately improves motion evaluation performance by finding and moving the optimal position from which a joint can be accurately extracted. The embodiment can provide a high-level coaching service by effectively creating a situation in which a professional trainer accurately guides a learner's posture while moving to a position where observation is easy.

At this time, the human motion evaluation apparatus 100 may be implemented integrally with the mobile robot, or a separate unit capable of controlling the movement of the mobile robot through wireless communication with the mobile robot and receiving an image captured by the mobile robot. may be implemented as a device.

In the following embodiments, for convenience of description, the human motion evaluation apparatus 100 is mounted on a mobile robot and implemented as an integrated unit, but the present invention is not limited thereto.

3 is a flowchart illustrating a method for evaluating a human motion using a mobile robot according to an embodiment.

Referring to FIG. 3 , the mobile robot 100 identifies a human motion based on an image obtained through an installed camera (S110).

In this case, the exercise operation may be an operation that a human consciously performs to improve health.

The mobile robot 100 determines what action the human is currently taking by analyzing an image sequence captured from a video obtained through a camera.

At this time, the human may take a daily action or may perform an exercise motion that is an evaluation target according to the embodiment.

The mobile robot 100, when a human is performing an exercise action other than a daily action, identifies the type of the exercise action.

In this case, a specific method for identifying the motion may follow the method described in the patent “User Adaptive Behavior Recognition Method of Robot and Apparatus therefor” (Application No. 10-2019-0109364).

At this time, the mobile robot 100 searches for information corresponding to the identified exercise motion from a database in which standard postures and posture profiles corresponding to each exercise motion type are constructed.

At this time, the standard posture may be correct posture information for the exercise motion.

In this case, the posture profile may be meta information including segment weight information and optimal viewing direction information for each segment. In this case, the segment weight information may be segment information to be evaluated as important in performing a standard posture, and the optimal viewing direction information for each segment may be optimal viewing direction information for accurately evaluating each segment.

These standard postures and posture profiles for each type of exercise may be constructed with information obtained from a corresponding exercise expert.

The mobile robot 100 evaluates the accuracy of the posture by comparing the posture of the human in the image captured at one position with the standard posture corresponding to the exercise motion (S120).

At this time, the human motion evaluation apparatus 200 extracts an evaluation target segment from the human posture based on the posture profile, evaluates posture accuracy through comparison of the evaluation target segment and the corresponding segment in the standard posture, and evaluates the evaluation target segment. The posture accuracy is stored in the posture evaluation diagram. A detailed description thereof will be described later with reference to FIGS. 4 and 5 .

Then, the mobile robot 100 determines whether it needs to move to another location (S130). That is, it is determined whether the posture evaluation has been sufficiently performed at one position.

When it is determined that it needs to move as a result of the determination in S130, the mobile robot 100 moves to the next target position (S140).

In this case, the next target position may be an advantageous position for posture evaluation. A detailed description of S140 will be described later with reference to FIGS. 6 to 10 .

Through S120 to S140, the mobile robot 100 repeatedly performs position movement and posture evaluation, and information on the joints detected at each position and the evaluated segments is continuously stored in the posture evaluation diagram.

On the other hand, when it is determined that movement is not necessary as a result of the determination in S130, the mobile robot 100 integrates the posture evaluation results and finally provides an evaluation result for human motion (S150). That is, when the posture evaluation information is secured for all segments shown in the posture evaluation diagram, the mobile robot 100 finally evaluates the human motion by integrating the posture evaluation information obtained so far.

At this time, the final motion evaluation is based on the weight w of each segment provided in the posture profile, the detection accuracy j of each joint calculated in S120, the posture accuracy score s of each segment, and the number of evaluations n of each segment. The posture accuracy score M is determined by the following <Equation 1>.

<Equation 1>

In <Equation 1>, m may be the number of all segments, for example, 15, and Bi is the final posture accuracy score of the i-th segment, which is a summation of posture accuracy scores when the i-th segment is evaluated n times. It can be calculated by <Equation 2>.

<Equation 2>

In <Equation 2>, Ei is a segment detection accuracy value calculated by the detection accuracy of both end joints ja and jb constituting the i-th segment, and may be calculated by the following <Equation 3>.

<Equation 3>

Then, the posture evaluation step (S120) shown in FIG. 3 will be described in detail.

4 is a flowchart illustrating a posture evaluation step according to an embodiment, and FIG. 5 is an exemplary view of a posture evaluation diagram according to an embodiment.

Referring to FIG. 4 , the mobile robot 100 captures an image including a human's entire body at a current location (S210).

Then, the mobile robot 100 detects the positions of the joints in the captured human's whole body image (S220).

Referring to the posture evaluation diagram shown in FIG. 5 , a total of 16 joints from j1 to j16 that can be detected in the whole body image of a human may be included. Also, a total of 15 segments from s1 to s15 may be configured between the total of 16 joints.

At this time, joints that are clearly visible without occluding in the view of the direction in which the mobile robot 100 looks at the human from the current position can be detected from the captured whole body image, so the detection accuracy is high. However, since joints invisible in the robot view cannot be detected from the whole body image, the mobile robot 100 must estimate the location of the corresponding joint. However, these estimated joint positions are inevitably less accurate than the detected joint positions.

For example, if the mobile robot 100 is located in the right direction of the human, the detection accuracy of the joints of the right arm and the right leg shown in the robot view is high, but the joints of the left arm and the left leg are invisible due to being hidden from the robot view. have low detection accuracy.

Therefore, according to an embodiment, the mobile robot 100 performs a posture evaluation process only for joints whose detection accuracy is greater than or equal to a reference value in the robot view.

That is, the mobile robot 100 calculates the joint detection accuracy for each of the detected joints, and determines whether a joint having a calculated joint detection accuracy equal to or higher than a reference value exists (S230).

As a result of the determination in S230, if there is a joint with a joint detection accuracy equal to or greater than the reference value, the mobile robot 100 calculates a posture accuracy score only for a body segment composed of the corresponding joints (S250).

In this case, one segment may be evaluated at least once.

Accordingly, the mobile robot 100 further performs a step of determining whether it is necessary to repeatedly evaluate segments composed of joints having detected joint detection accuracy greater than or equal to a reference value (S240). That is, it is determined whether sufficient repeated evaluation has been performed for the corresponding segment.

Therefore, as a result of the determination in S240, if repeated evaluation of the segment composed of the corresponding joint is required, the mobile robot 100 calculates the posture accuracy score of the corresponding segment (S250), and updates the evaluation result when the posture accuracy score is calculated (S250). S260).

Then, when the posture accuracy score of the segment is calculated, the mobile robot 100 updates the evaluation result (S260).

At this time, the posture accuracy score may be recorded/updated in the posture evaluation diagram as shown in FIG. 5 .

At this time, in the posture evaluation diagram, information calculated by the above-described posture evaluation, such as joint detection accuracy, segment posture accuracy score, and segment evaluation count, may be recorded.

In this case, since the same joint and segment can be evaluated multiple times, all information calculated during each evaluation can be recorded.

In this case, the number of evaluations may be displayed by distinguishing them by brightness or color difference.

For example, referring to FIG. 5 , brightness displayed on a joint or segment may be proportional to the number of evaluations. That is, segment s4 may be evaluated 0 times, segment s14 may be evaluated once, segment s5 may be evaluated twice, and segment s8 may be evaluated three times.

This posture evaluation diagram can also be used as an interface for providing information on the current evaluation progress state to the user.

On the other hand, referring to FIG. 4 again, as a result of the determination of S230 and S240, even if there is no joint with a joint detection accuracy of the reference value or more at the current position, or even if there is a joint with a reference value or more, the segment composed of the corresponding joint is sufficiently evaluated more than the set number of times. If so, the mobile robot 100 moves to the next position (S130).

Next, the robot position movement step (S140) shown in FIG. 3 will be described in detail.

6 is a flowchart for explaining a step of moving a robot position according to an embodiment, FIG. 7 is an exemplary view of a posture profile for setting a target position according to an embodiment, and FIG. 8 describes a step of setting a target position according to an embodiment. 9 is an exemplary diagram for explaining a method for searching for an optimal viewing direction for a segment according to an embodiment, and FIG. 10 is an exemplary diagram for determining a target position of a mobile robot when a posture profile is not provided according to an embodiment. am.

Referring to FIG. 6 , the mobile robot 100 sets a target position to move to next (S310).

At this time, the target position may be an advantageous position for posture evaluation.

At this time, setting the target position may be performed in various ways depending on whether or not the posture profile exists.

In this case, the posture profile may include a weight value indicating importance of each segment in evaluating a specific posture and a 2D direction vector indicating an easy direction for evaluating each segment. For example, referring to FIG. 7 , weights w1, w2, ..., w15 and a 2-dimensional direction vector may be displayed on each segment of the posture profile.

Such a posture profile may be obtained from an expert in the corresponding exercise motion, and a domain expert may easily know which segment is most important in taking a corresponding posture and information on the optimal observation direction for evaluating that segment through experience. Because.

For example, in the case of a squat posture, if domain experts determine that the calf and thigh segments are the most important and the spine posture is next important, weights of the corresponding segments may be set high according to the degree of importance. On the other hand, since arm posture is relatively less important, the weights of corresponding segments can be set low, and in the case of segments that are not important at all, they can be set to 0. In addition, the distance between the legs, which is an important evaluation factor in the squat posture, is easiest to check from the front, and the degree of knee flexion and whether or not the spine is properly straightened can be most accurately evaluated when observed from the side.

The domain knowledge possessed by these domain experts can be easily provided because it can be easily edited at no cost through the posture profile user interface.

However, in consideration of a situation in which posture profile information is not provided, in an embodiment, a target movement position may be set depending on whether or not information provided from a posture profile or whether an optimal viewing direction is included in the posture profile is present.

Specifically, referring to FIG. 8 , the mobile robot 100 determines whether a posture profile exists (S311).

As a result of the determination in S311, if the posture profile exists, the mobile robot 100 determines whether to provide the optimal observation direction to the posture profile (S312).

As a result of the determination of S311 and S312, when segment importance and optimal observation direction information are provided in the posture profile, the mobile robot 100 performs evaluation in order of high segment importance, which is the most important information in determining the target movement position of the robot ( S313) However, the target movement position is set based on the optimal viewing direction information of each segment provided in the posture profile (314).

On the other hand, as a result of the determination in S311 and S312, when only the importance of the segment is provided in the posture profile, the mobile robot 100 performs evaluation in the order of highest segment importance (S315), but since optimal observation direction information is not provided, the corresponding segment The best observable position, that is, the optimal observation position should be detected (S316).

As shown in FIG. 9 , assume that a left thigh segment AB composed of joints A and B is evaluated in evaluating a specific posture. At this time, the optimal position for the robot to evaluate the segment AB is a position that satisfies both of the following situations.

First, the optimal observation position should be the position where joint A and joint B can be most accurately detected. As described above, joint detection accuracy is very important in posture evaluation. Therefore, joint A and joint B must be well observed without occlusion and detection with high accuracy.

Second, the angle formed by segment AB should be the position where the robot can best observe. The posture evaluation is evaluated as a set of angles between corresponding segments of the user when the user assumes a specific posture and corresponding segments of the standard posture. Therefore, selection of a position where the angle formed by the segment can be accurately measured is as important as accurate joint detection.

A position of a robot that satisfies both of the above two conditions is a point that simultaneously satisfies the following three conditions.

① The point where both points A and B are observed in the image captured by the robot

② The point where line segment AB composed of two points A and B on 3D is perpendicular to the z-axis of the 3D coordinate system centered on the robot (more precisely, the robot camera)

③ The point where d1, the measured distance in the z direction from the robot to point A, and d2, the measured distance in the z direction from the robot to point B, are the same

In FIG. 9 , the position of the mobile robot satisfying all of the above three conditions may be R1. Both joints A and B can be observed in the 2D image consisting of the xy plane captured by the mobile robot at position R1. At the same time, segment AB is perpendicular to the z-axis of the mobile robot, and is a point where the distance from the mobile robot to joint A and joint B are the same.

Therefore, the position R1 that satisfies all of the above three conditions can be set as an optimal observation point where the joint can be best detected and the angle formed by the segment can be best observed.

On the other hand, position R2 satisfies conditions ② and ③, but since the segment AB is covered by the user's right thigh, it is not an optimal movement position because condition ① is not satisfied.

However, it is not easy to find a point that accurately satisfies the above conditions ② and ③ in a situation where a mobile robot is operated. Therefore, in the embodiment, in the case of condition ②, the condition that the angle formed by the line segment AB and the z-axis must be perpendicular can be relaxed and the designated section setting value (90 degrees ± α) can be used. Also, in the case of condition ③, the condition that d1 and d2 must be the same can be relaxed and the specified distance difference setting value (|d1-d1|<α) can be used.

On the other hand, when performing a specific posture, each segment may have several optimal positions depending on the case. In this case, the target position closest to the current robot position can be set as the next robot position.

Referring back to FIG. 8 , when the posture profile is not provided as a result of the determination in step S311, the mobile robot 100 determines that all segments have the same importance. Accordingly, the mobile robot 100 sets a position obtained by dividing a concentric circle at a predetermined distance from the human at a predetermined angle as a target position so as to capture the human body (S317). For example, referring to FIG. 10 , the mobile robot 100 may repeatedly perform posture evaluation at a corresponding position while moving sequentially in eight directions while maintaining a certain distance from a human.

Referring back to FIG. 6 , the mobile robot 100 moves toward the target position of the robot through the control of the hardware driving unit of the robot (S320). At this time, the human may move while maintaining an appropriate distance from the user so that the whole body of the human may be present in the image.

The mobile robot 100 determines whether or not it has arrived at the target position while continuously determining the current position while moving (S330). The mobile robot 100 repeatedly performs S320 and S330 until it is determined that it has arrived at the target location (S340).

On the other hand, if it is determined that the robot has arrived at the target position (S340), it performs the posture evaluation process of S120 (see FIG. 4).

11 is a diagram showing a configuration of a computer system according to an embodiment.

The human posture evaluation apparatus using the mobile robot according to the embodiment may be implemented in the computer system 1000 such as a computer-readable recording medium.

Computer system 1000 may include one or more processors 1010, memory 1030, user interface input devices 1040, user interface output devices 1050, and storage 1060 that communicate with each other over a bus 1020. can In addition, computer system 1000 may further include a network interface 1070 coupled to network 1080 . The processor 1010 may be a central processing unit or a semiconductor device that executes programs or processing instructions stored in the memory 1030 or the storage 1060 . The memory 1030 and the storage 1060 may be storage media including at least one of volatile media, nonvolatile media, removable media, non-removable media, communication media, and information delivery media. For example, memory 1030 may include ROM 1031 or RAM 1032 .

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art can implement the present invention in other specific forms without changing its technical spirit or essential features. You will understand that there is Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting.

Claims (20)

Analyzing a whole body image of a human captured by a camera mounted on a mobile robot to identify an exercise motion of the human, and detecting a standard posture and posture profile corresponding to the motion motion;
Evaluating a posture of the human by comparing a standard posture corresponding to the identified exercise motion with whole body images of the human captured at at least two or more target positions by a camera of the mobile robot; and
Comprehensive evaluation of the human's exercise motion based on the human's posture evaluation information in each of at least two or more target positions,
The posture profile is
It includes segment weight information, which is segment information to be evaluated as important in performing a standard posture,
target position,
It is set based on the optimal observation direction of at least one segment selected based on segment weight information as an evaluation target when performing a standard posture, and two joints constituting the selected segment are observed, extending vertically from the center of the segment, and two joints The distance from each is set to the same point,
According to the result of evaluating the human posture based on the image taken at one position, the next target position to which the mobile robot should move is set,
At the stage of evaluating the human posture,
The postural accuracy of the segments composed of the joints detected in the human whole body image is repeatedly evaluated for each segment,
The detection accuracy of the joint calculated by the posture evaluation, the posture accuracy score of the segment, and the number of evaluations of the segment are recorded in the posture evaluation diagram,
The step of comprehensively evaluating the human's motor motion,
Comprehensive evaluation based on the posture evaluation diagram and the posture profile corresponding to the identified exercise motion,
calculating segment detection accuracy as an average of detection accuracies of joints constituting the segment;
calculating a final posture accuracy score obtained by synthesizing a value obtained by multiplying the segment detection accuracy and the posture accuracy score of the segment according to the number of evaluations of the segment; and
A method for evaluating human motion using a mobile robot, comprising calculating a final motion accuracy based on a final posture accuracy score of each segment and a weight of each segment included in a posture profile. The method of claim 1, wherein the step of evaluating the human posture,
Detecting a joint in a human body image; and
A method for evaluating human motion using a mobile robot, comprising calculating a posture accuracy score for a segment composed of joints having a detection accuracy equal to or greater than a reference value. The method of claim 2, wherein the step of evaluating the human posture,
Further comprising recording the detection accuracy of the joint and the posture accuracy score for the segment in the posture evaluation diagram,
posture evaluation,
A method for evaluating human motion using a mobile robot, in which all segments and joints of a human are displayed. The method of claim 3, wherein the step of evaluating the human posture,
Segment-by-segment evaluation is repeated,
The step of recording in the posture evaluation diagram is,
A method for evaluating human motion using a mobile robot, wherein the number of evaluations of segments is recorded in a posture evaluation diagram so as to be distinguished according to color or brightness. The method of claim 2, wherein the step of evaluating the human posture,
Set the target position based on the posture profile,
posture profile,
When the 2D direction vector for each of the segments is further included, the next segment to be observed is selected in the order of weight, and the target position, which is the optimal observation position of the segment, is set based on the 2D direction vector of the selected segment. Human motion evaluation method using robot. delete The method of claim 2, wherein the step of evaluating the human posture,
If no posture profile is provided,
A method for evaluating human motion using a mobile robot, in which a target position is set while moving a point at a predetermined distance from the human by a predetermined angle. delete a memory in which at least one program is recorded; and
A processor that executes a program;
program,
Analyzing a whole body image of a human captured by a camera mounted on a mobile robot to identify an exercise motion of the human, and detecting a standard posture and posture profile corresponding to the motion motion;
Evaluating a posture of the human by comparing a standard posture corresponding to the identified exercise motion with whole body images of the human captured at at least two or more target positions by a camera of the mobile robot; and
Performing a comprehensive evaluation of the human's exercise motion based on the human's posture evaluation information in each of at least two or more target positions;
The posture profile is
It includes segment weight information, which is segment information to be evaluated as important in performing a standard posture,
The target position is set based on the optimal observation direction of at least one segment selected based on segment weight information as an evaluation target when performing a standard posture, two joints constituting the selected segment are observed, and vertically extended from the center of the segment. , the distance from each of the two joints is set to the same point,
According to the result of evaluating the human posture based on the image taken at one position, the next target position to which the mobile robot should move is set,
At the stage of evaluating the human posture,
The postural accuracy of the segments composed of the joints detected in the human whole body image is repeatedly evaluated for each segment,
The detection accuracy of the joint calculated by the posture evaluation, the posture accuracy score of the segment, and the number of evaluations of the segment are recorded in the posture evaluation diagram,
The step of comprehensively evaluating the human's motor motion,
Comprehensive evaluation based on the posture evaluation diagram and the posture profile corresponding to the identified exercise motion,
calculating segment detection accuracy as an average of detection accuracies of joints constituting the segment;
calculating a final posture accuracy score obtained by synthesizing a value obtained by multiplying the segment detection accuracy and the posture accuracy score of the segment according to the number of evaluations of the segment; and
A human motion evaluation apparatus using a mobile robot, comprising: calculating a final motion accuracy based on a final posture accuracy score of each segment and a weight of each segment included in a posture profile. The method of claim 9, wherein the step of evaluating the human posture,
Detecting a joint in a human body image; and
An apparatus for evaluating human motion using a mobile robot, comprising calculating a posture accuracy score for a segment composed of joints having a detection accuracy equal to or greater than a reference value. The method of claim 10, wherein the step of evaluating the human posture,
Further comprising recording the detection accuracy of the joint and the posture accuracy score for the segment in the posture evaluation diagram,
posture evaluation,
An apparatus for evaluating human motion using a mobile robot, in which all segments and joints of a human are displayed. The method of claim 11, wherein the step of evaluating the human posture,
Segment-by-segment evaluation is repeated,
The step of recording in the posture evaluation diagram is,
A human motion evaluation device using a mobile robot, which records the number of evaluations of segments in a posture evaluation diagram so as to be distinguished according to color or brightness. The method of claim 10, wherein the step of evaluating the human posture,
Set the target position based on the posture profile,
If the posture profile further includes a two-dimensional direction vector,
An apparatus for evaluating human motion using a mobile robot, which selects the next segment to be observed in order of weight and sets a target position, which is an optimal observation position for the segment, based on a 2-dimensional direction vector of the selected segment. delete The method of claim 9, wherein the step of evaluating the human posture,
An apparatus for evaluating human motion using a mobile robot, which sets a target position while moving a point at a predetermined distance from a human by a predetermined angle when a posture profile is not provided. delete Analyzing a human body image captured by a camera mounted on a mobile robot to identify a movement motion of the human;
detecting a standard posture and posture profile corresponding to the identified exercise motion;
setting a target position based on the posture profile;
Evaluating a posture of a human by comparing a full-body image of a human photographed by a camera with a standard posture after the mobile robot moves to a set target position;
Comprehensive evaluation of the human's exercise motion based on the human's posture evaluation information at each target position,
The step of setting the target position and the step of evaluating the human posture are repeatedly performed,
The posture profile is
It includes segment weight information, which is segment information to be evaluated as important in performing a standard posture,
The target position is set based on the optimal observation direction of at least one segment selected based on segment weight information as an evaluation target when performing a standard posture, two joints constituting the selected segment are observed, and vertically extended from the center of the segment. , the distance from each of the two joints is set to the same point,
According to the result of evaluating the human posture based on the image taken at one position, the next target position to which the mobile robot should move is set,
At the stage of evaluating the human posture,
The postural accuracy of the segments composed of the joints detected in the human whole body image is repeatedly evaluated for each segment,
The detection accuracy of the joint calculated by the posture evaluation, the posture accuracy score of the segment, and the number of evaluations of the segment are recorded in the posture evaluation diagram,
The step of comprehensively evaluating the human's motor motion,
Comprehensive evaluation based on the posture evaluation diagram and the posture profile corresponding to the identified exercise motion,
calculating segment detection accuracy as an average of detection accuracies of joints constituting the segment;
calculating a final posture accuracy score obtained by synthesizing a value obtained by multiplying the segment detection accuracy and the posture accuracy score of the segment according to the number of evaluations of the segment; and
A method for evaluating human motion using a mobile robot, comprising calculating a final motion accuracy based on a final posture accuracy score of each segment and a weight of each segment included in a posture profile. The method of claim 17, wherein the step of evaluating the human posture,
Detecting a joint in a human body image; and
A method for evaluating human motion using a mobile robot, comprising repeatedly calculating a posture accuracy score for a segment composed of joints having a detection accuracy equal to or greater than a reference value by a predetermined number of evaluations. The method of claim 17, wherein setting the target location comprises:
Set the target position based on the posture profile,
posture profile,
When a 2D direction vector is further included, a human operation using a mobile robot selects the next segment to be observed in the order of weight and sets a target position, which is the optimal observation position of the segment, based on the 2D direction vector of the selected segment. Assessment Methods. delete

Images