WO2023013562A1 - Fatigue estimation system, fatigue estimation method, posture estimation device, and program - Google Patents

Abstract

A fatigue estimation system (200) comprises: a plurality of imaging devices (201) that individually output images in which different parts of the body of a subject (11) are imaged; a posture estimation device (posture estimation unit (105)) that estimates the posture of the subject (11) on the basis of a plurality of images separately outputted by the plurality of imaging devices (201); and a fatigue estimation device (100) that estimates and outputs a level of fatigue of the subject (11) on the basis of the estimation result of the posture of the subject (11).

Description

Fatigue estimation system, fatigue estimation method, posture estimation device and program

The present disclosure relates to a fatigue estimation system for estimating the degree of fatigue of a subject, a posture estimation device used in the estimation system, a fatigue estimation method, and a program.

In recent years, there have been cases where accumulated fatigue leads to poor physical condition, injuries and accidents. On the other hand, attention has been focused on technology for preventing poor physical condition, injuries, accidents, etc. by estimating the degree of fatigue. For example, as a fatigue estimation system for estimating the degree of fatigue, Patent Document 1 discloses a fatigue determination device that determines the presence or absence of fatigue and the type of fatigue based on force measurement and bioelectrical impedance measurement. .

JP 2017-023311 A

By the way, in the conventional fatigue determination device and the like exemplified in Patent Document 1, there are cases where the estimated posture is not appropriate and the degree of fatigue cannot be estimated. Therefore, the present disclosure provides a fatigue estimation system and the like that more appropriately estimate the posture and estimate the degree of fatigue.

A fatigue estimation system according to one aspect of the present disclosure includes a plurality of imaging devices that respectively output images in which different parts of the subject's body part are captured, and the plurality of imaging devices. a posture estimation device for estimating the posture of the subject based on the plurality of images; a fatigue estimation device for estimating and outputting the degree of fatigue of the subject based on the result of estimating the posture of the subject; Prepare.

A posture estimation device according to an aspect of the present disclosure is the posture estimation device described above.

Further, a fatigue estimation method according to an aspect of the present disclosure is a fatigue estimation method executed by a fatigue estimation device, wherein a plurality of and obtaining the images from each of the imaging devices, one image obtained by imaging one body part among the plurality of images, and the one body part and at least one joint among the plurality of images estimating the joint positions of the body parts including the one body part and the other body parts as the posture of the subject, and Based on the result of estimating the posture of the subject, the degree of fatigue of the subject is estimated and output to an output device connected to the fatigue estimation device.

A program according to one aspect of the present disclosure is a program for causing a computer to execute the fatigue estimation method described above.

A fatigue estimation system or the like according to one aspect of the present disclosure can more appropriately estimate the posture and estimate the degree of fatigue.

FIG. 1A is a first diagram for explaining estimation of fatigue level according to the embodiment. FIG. 1B is a second diagram for explaining estimation of fatigue level according to the embodiment. FIG. 1C is a third diagram for explaining estimation of fatigue level according to the embodiment. FIG. 2 is a block diagram showing the functional configuration of the fatigue estimation system according to the embodiment. FIG. 3 is a flowchart showing a fatigue level estimation method according to the embodiment. 4A is a diagram showing a subject standing still in Posture A. FIG. 4B is a diagram showing a subject standing still in Posture B. FIG. FIG. 5A is a first diagram illustrating accumulation of estimated fatigue level of a subject according to the embodiment. FIG. 5B is a second diagram illustrating accumulation of estimated fatigue level of the subject according to the embodiment. FIG. 6 is a diagram illustrating an example of synthesizing joint positions according to the embodiment. FIG. 7 is a diagram showing a display example of estimation results according to the embodiment.

Hereinafter, embodiments will be specifically described with reference to the drawings. It should be noted that the embodiments described below are all comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements not described in independent claims will be described as optional constituent elements.

It should be noted that each figure is a schematic diagram and is not necessarily strictly illustrated. Moreover, in each figure, the same code|symbol is attached|subjected with respect to substantially the same structure, and the overlapping description may be abbreviate|omitted or simplified.

(Embodiment)
[overview]
In the fatigue estimation system 200 (see FIG. 2 described later) in the present embodiment, the posture is estimated by capturing an image of the subject 11 (see FIG. 1A described later) whose fatigue level is to be estimated. The degree of fatigue of the subject 11 is estimated from the estimated posture. Here, depending on the positional relationship between the subject 11 and the imaging device 201 (see FIG. 1A to be described later) for acquiring an image, there may be an obstacle or the like between them. Obtaining the desired image of the subject 11 may be difficult. As a result, in some cases, the fatigue level of the subject 11 cannot be estimated.

Therefore, in the present disclosure, by using a plurality of imaging devices 201 each capable of imaging at least part of the subject 11, in the image of the subject 11 necessary for estimating the posture of the subject 11 Fill in the missing parts. This provides the fatigue estimation system 200 that enables estimation of the degree of fatigue of the subject 11 even when the posture of the subject 11 cannot be estimated with one imaging device 201 . In addition, although the details will be described later, in the present disclosure, in order to estimate the fatigue of the subject 11, the load on at least one of the muscles and joints and the deteriorating blood flow are estimated from the estimated posture of the subject 11.

Therefore, in the present disclosure, the posture estimation device used for posture estimation can also be used for estimating muscle load, joint load, and the degree of deterioration of blood flow. In addition, the posture estimation device according to the present disclosure can be applied as a device for estimating the posture of the subject 11 for various uses. That is, the posture estimation apparatus of the present disclosure can obtain an appropriate posture of the subject 11 even from images captured by the imaging devices 201 arranged such that only a part of the body of the subject 11 is captured by each of the imaging devices 201. Information (2D skeletal information, 3D skeletal information, and feature quantities such as the angle between the neck and the spine and the angle between the spine and the lower leg) can be generated and applied to all uses. be.

[Fatigue estimation system]
The overall configuration of the fatigue estimation system 200 according to the embodiment will be described below. FIG. 1A is a first diagram for explaining estimation of fatigue level according to the embodiment. FIG. 1B is a second diagram for explaining estimation of fatigue level according to the embodiment. FIG. 1C is a third diagram for explaining estimation of fatigue level according to the embodiment.

In the embodiment, the fatigue estimation system 200 in the present disclosure is a system that estimates the degree of fatigue of the subject 11 using an image output by imaging the subject 11 using the imaging device 201 . The imaging device 201 is not limited in its form as long as it is a camera that captures the subject 11 and outputs an image, and as shown in FIG. is realized by

Here, the subject is in a posture of sitting on the chair 12. In the fatigue estimation system 200 according to the present disclosure, the degree of fatigue of the subject 11 is estimated based on the fatigue accumulated by taking a static posture with a fixed posture among the fatigue of the subject 11 . In other words, this estimates the fatigue accumulated by the load on at least one of muscles and joints and deteriorating blood flow (hereinafter also referred to as decreased blood flow) due to a fixed posture. Accordingly, the subject 11 is in a static posture, sitting, lying down or standing still for at least a certain period of time. The fixed period is, for example, a minimum period during which fatigue can be estimated in the fatigue estimation system 200, such as several tens of seconds or several seconds. Such a period is determined depending on the processing capabilities of the imaging device 201 and the fatigue estimation device 100 (see FIG. 2 described later) that configure the fatigue estimation system 200 .

Examples of subjects 11 who take such a stationary posture include desk workers in offices, drivers who steer moving bodies, people who perform muscle strength training using a load in a stationary posture, residents of facilities such as hospitals, airplanes, and the like. passengers and crew members.

An image captured and output by the imaging device 201 is processed by the estimating device 100 to estimate the posture of the subject 11 (for example, the joint position 11a) as shown in FIG. 1B. The estimated posture of the subject 11 is output as a rigid body link model as an example. Specifically, as shown in FIG. 1B, the straight skeletons are connected by joints indicated by black dots, and the posture of the subject 11 can be reproduced by the positional relationship between the two skeletons connected by one joint. The posture is estimated by image recognition, and output as the rigid body link model described above based on the positional relationship between the joints.

By applying the estimated rigid body link model to the musculoskeletal model 11b shown in FIG. 1C, each body part of the muscles that pull the skeletons together and the joints that connect the skeletons so that the positional relationship can be changed. In order to maintain the positional relationship according to the estimated posture, the amount of load applied to at least one of the muscles and joints of each body part is calculated as an estimated value. Since the estimated value of the load on at least one of the muscles and joints of each body part is accumulated as the duration of the stationary posture increases, calculation using the estimated value of the load and the duration The degree of fatigue due to the object person 11 maintaining a still posture is calculated by . In the following description, "at least one of muscles and joints" is also expressed as "muscles and/or joints."

In addition, in the present embodiment, it is possible to estimate the degree of fatigue based on the estimated value of the blood flow of the subject 11 in addition to the estimated value of the load applied to the muscles and/or joints. In the following description, an example of estimating the fatigue level of the subject 11 using the estimated values of the load on the muscles and the load on the joints will be mainly described. It is also possible to estimate the degree of fatigue of the subject 11 with higher accuracy. Furthermore, the fatigue level of the subject 11 can also be estimated using an estimated value of any one of the amount of load on the muscles of the subject 11, the amount of load on the joints, and the amount of blood flow. In addition, in the present disclosure, the estimation of the fatigue level of the subject 11 based on the posture of the subject 11 is not limited to the above example, and any existing technique for estimating the fatigue level can be applied.

As an example, in the present embodiment, after estimating the posture of the subject 11, the fatigue estimation system 200, based on the duration of the posture, the amount of load on the muscles of the subject 11, the amount of load on the joints, and at least one of blood flow. The fatigue estimation system 200 estimates the degree of fatigue of the subject 11 based on the estimated value of at least one of the estimated muscle load, joint load, and blood flow of the subject 11 . Hereinafter, for the sake of simplification, the estimated value of the load amount may be simply referred to as the load amount or the estimated value. In addition, when the estimated value of the blood flow is included in the estimated value, the load is replaced with the blood flow, a large load is replaced with a decrease in blood flow, and a small load is replaced with an increase in blood flow. may

In addition, as described above, the blood flow is information for quantifying the blood flow that deteriorates when the subject 11 maintains the posture. As the blood flow decreases, it means that the blood flow of the subject 11 is worsening, and can be used as an index of fatigue caused by the deterioration of the blood flow. The blood flow may be obtained as an absolute numerical value at the time of measurement, or may be obtained as a relative numerical value between two different time points. For example, the degree of deterioration of the blood flow of the subject 11 can be estimated from the posture of the subject 11 and the relative numerical values of the blood flow at two points of time when the posture starts and ends. In addition, since there is a correlation between the posture of the subject 11 and the duration of the posture, and the deterioration of blood flow, the blood flow rate of the subject can be calculated simply from the posture of the subject 11 and the duration of the posture. can be estimated.

Further, in the following description, at least one of the amount of load on the muscles, the amount of load on the joints, and the amount of blood flow is estimated from the posture of the subject 11 using the musculoskeletal model described above. In addition to the musculoskeletal model described above, a method using actual measurement data can also be applied as a method for estimating the load on muscles, the load on joints, and the blood flow. This measured data is a database constructed by accumulating measured values of load on muscles, load on joints, and blood flow, which are measured for each posture, in association with the posture. In the fatigue estimation system 200 in this case, by inputting the estimated posture of the subject 11 into the database, the measured values of the load on the muscles, the load on the joints, and the blood flow in the corresponding posture are obtained. can be obtained as output.

The actual measurement data may be constructed using actual measurement values for each individual in consideration of individual differences in the subject 11, and statistical analysis or machine learning is performed for big data obtained from an unspecified number of subjects. It may be qualified and constructed so as to match each subject 11 by analysis processing such as.

Next, the functional configuration of the fatigue estimation system 200 according to the present disclosure will be described using FIG. FIG. 2 is a block diagram showing the functional configuration of the fatigue estimation system according to the embodiment.

As shown in FIG. 2, the fatigue estimation system 200 in the present disclosure includes a fatigue estimation device 100, multiple imaging devices 201, a display device 205, and a recovery device 206.

Estimation device 100 includes acquisition section 101 , identification section 102 , posture estimation section 105 , first calculation section 106 , second calculation section 107 , fatigue estimation section 108 and output section 109 .

The acquisition unit 101 is a communication module that is connected to each of the plurality of imaging devices 201 and acquires images of the subject 11 captured from each of the plurality of imaging devices 201 . The connection between the acquisition unit 101 and the imaging device 201 is performed by wire or wirelessly, and the method of communication performed through the connection is not particularly limited.

The fatigue estimation device 100 may include, in addition to the imaging device 201 described above, a communication module that is connected to a timer and acquires time from the timer.

In addition, the fatigue estimation device 100 may also include a communication module that is connected to the pressure sensor and acquires the pressure distribution from the pressure sensor.

In addition, the fatigue estimation device 100 may also include a communication module that is connected to the reception device and acquires personal information from the reception device.

The identification unit 102 is a processing unit realized by executing a predetermined program using a processor and memory. The identification unit 102 is installed to realize an identification function for identifying one target person 11 from other persons. Details of the function of the identification unit 102 will be described later.

The posture estimation unit 105 is a processing unit realized by executing a predetermined program using a processor and memory. By the processing of the posture estimation unit 105, the posture of the subject 11 is estimated based on the image acquired by the acquisition unit 101, the additionally acquired pressure distribution, and the like. Posture estimation section 105 is an example of a posture estimation device. That is, the posture estimation apparatus according to the present embodiment is implemented as posture estimation section 105 incorporated in fatigue estimation apparatus 100 .

The first calculation unit 106 is a processing unit realized by executing a predetermined program using a processor and memory. Based on the estimated posture of the subject 11 and additionally acquired personal information, the load amount applied to each muscle and/or joint is calculated by the processing of the first calculation unit 106 .

The second calculation unit 107 is a processing unit realized by executing a predetermined program using a processor and memory. By the processing of the second calculation unit 107, the amount of recovery from fatigue in each muscle and/or joint is calculated based on the amount of change in the estimated change in posture of the subject 11. FIG.

The fatigue estimation unit 108 is a processing unit realized by executing a predetermined program using a processor and memory. The fatigue estimating unit 108 uses the posture estimated by the posture estimating unit 105 and the time obtained from a timing device or an internal oscillator to estimate fatigue of the subject 11 based on the duration of the posture. Estimate degrees.

The output unit 109 is a communication module that is connected to the display device 205 and the recovery device 206 and outputs to the display device 205 and the recovery device 206 the content based on the fatigue level estimation result by the fatigue estimation device 100 . The connection between the output unit 109 and the display device 205 or the recovery device 206 is performed by wire or wirelessly, and the method of communication performed through the connection is not particularly limited.

In addition, when the posture estimation unit 105 fails to estimate the posture of the target person 11, the output unit 109 has a function of notifying the outside that there is a target person 11 whose degree of fatigue cannot be estimated. also have That is, the output unit 109 outputs the estimation result of the posture estimation unit 105 indicating that estimation of the posture of the subject 11 has failed.

At this time, the output unit 109 outputs the above estimation result as notification information for notifying the existence of the target person 11 whose posture estimation has failed. The administrator or the like of the fatigue estimation system 200 can take measures such as estimating the fatigue level of the subject 11 whose posture estimation has failed based on this notification information using another system, or directly listening to the fatigue level. be. In this manner, the output unit 109 is configured in advance so that the fatigue level of the target person 11 whose posture has failed to be estimated will not be missed.

As described above, the imaging device 201 is a device that captures an image of the subject 11 and outputs an image, and is realized by a camera. As the imaging device 201, an existing camera such as a security camera or a fixed-point camera may be used in the space where the fatigue estimation system 200 is applied, or a dedicated camera may be newly provided. Such an imaging device 201 is an example of an information output device that outputs an image as information about the position of the body part of the subject 11 . Therefore, the information to be output is an image, and is information including the positional relationship of the body parts of the subject 11 projected on the imaging device.

A clock is a device that measures time, and is realized by a clock. The time measured by the timer may be absolute time or relative elapsed time from a starting point. The timing device may be realized in any form as long as it can measure the time between two points of time when the target person 11 is detected to be still and the point of time when the degree of fatigue is estimated (that is, the duration of the stationary posture). .

A pressure sensor is a sensor having a detection surface, and measures the pressure applied to each of the unit detection surfaces that divide the detection surface into one or more. The pressure sensor thus measures the pressure for each unit detection surface and outputs the pressure distribution on the detection surface. The pressure sensor is provided so that the subject 11 is positioned on the detection surface.

For example, the pressure sensors are provided on the seat and backrest of the chair on which the subject 11 sits. Further, for example, the pressure sensor may have a marker attached on the detection surface, and the subject 11 may be guided onto the detection surface by a display such as "Please sit on the marker." By guiding the target person 11 onto the detection surface of the pressure sensor provided on a part of the floor in this way, the pressure sensor may output the pressure distribution of the target person 11 on the floor. Since the pressure distribution is used for the purpose of improving the accuracy of estimating the degree of fatigue, the fatigue estimation system 200 may be implemented without the pressure sensor if sufficient accuracy is ensured.

The reception device is a user interface that receives input of the personal information of the subject 11, and is realized by an input device such as a touch panel or keyboard. The personal information includes at least one of age, sex, height, weight, muscle mass, stress level, body fat percentage, and exercise proficiency. The age of the target person 11 may be a specific numerical value, or may be an age group divided by 10 years, such as teens, 20s, and 30s. The age range may be divided into two divisions based on a predetermined age, such as age and older, or may be other age groups.

In addition, the gender of the subject 11 is one suitable for the subject 11, which is selected from two of male and female. Also, as the height and weight, numerical values of the height and weight of the subject 11 are respectively accepted. Also, as the muscle mass, the muscle composition ratio of the subject 11 measured using a body composition meter or the like is accepted. In addition, the stress level of the subject 11 is selected by the subject 11 himself/herself from options such as high, medium, and low as the degree of subjective stress felt by the subject 11 .

Also, the body fat percentage of the subject 11 is the ratio of the body fat weight to the body weight of the subject 11, and is expressed, for example, as a percentage of 100.

In addition, the subject's 11 exercise proficiency may be quantified by a score obtained when the subject 11 executes a predetermined exercise program, or may be the status of the exercise that the subject 11 usually undertakes. The former is quantified by, for example, the time required to perform ten spins, the time required to run 50 meters, the flight distance of a long throw, and the like. The latter is quantified by, for example, how many days a week you exercise or how many hours you exercise. Since the personal information is used for the purpose of improving the accuracy of estimating the degree of fatigue, the fatigue estimation system 200 may be implemented without the reception device if sufficient accuracy is ensured.

The display device 205 is a device for displaying the content based on the fatigue level estimation results output by the output unit 109 . The display device 205 displays an image showing the content based on the result of estimating the degree of fatigue using a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The contents displayed by the display device 205 will be described later. Further, if the fatigue estimation system 200 is configured to only reduce the degree of fatigue of the subject 11 using the recovery device 206 for the subject 11, only the recovery device 206 may be provided, and the display device 205 Not required.

The recovery device 206 is a device that reduces the degree of fatigue of the subject 11 by promoting the subject's 11 blood circulation. Specifically, the recovery device 206 changes the arrangement of each part of the chair 12 by applying voltage, pressurizing, vibrating, heating, or the like, or by a mechanism provided in the chair 12, so that the sitting subject 11 Actively change the posture of As a result, the recovery device 206 changes the load on at least one of the muscles and joints of the subject 11 and promotes blood circulation. From the viewpoint of the blood flow, by promoting the blood circulation in this way, the influence of the deterioration of the blood flow due to the subject 11 being in the still posture is reduced, and the degree of fatigue is recovered. The recovery device 206 is pre-applied or contacted to the appropriate body part of the subject 11, depending on the configuration of the device.

It should be noted that when the blood circulation of the subject 11 is promoted by heating, the entire space around the subject 11 is heated. do not have. In addition, if the fatigue estimation system 200 is configured to display only the fatigue level estimation result to the subject 11, the display device 205 only needs to be provided, and the recovery device 206 is not essential.

[motion]
Next, estimation of the degree of fatigue of subject 11 using fatigue estimation system 200 according to the embodiment will be described with reference to FIGS. 3 to 6. FIG. FIG. 3 is a flowchart showing a fatigue level estimation method according to the embodiment.

The fatigue estimation system 200 first acquires the personal information of the subject 11 . Acquisition of personal information is performed by the target person 11 himself or a manager who manages the degree of fatigue of the target person 11 by inputting to the reception device. The input personal information of the subject 11 is stored in a storage device or the like (not shown), and is read out and used when estimating the degree of fatigue.

The fatigue estimation system 200 detects the subject 11 using the imaging device 201 . The detection of the target person 11 is performed by counting the persons within the angle of view so that all persons within the angle of view of the camera, which is the imaging device 201 , are the target persons 11 . That is, here, the operation of acquiring the number of people (A) to be detected is performed (S101). Next, among these persons, the number (B) of persons whose postures can be estimated without image processing, that is, with only one image is calculated (S102). Then, the fatigue estimation device 100 determines whether or not A and B match (S103). When the fatigue estimation apparatus 100 determines that A and B match (Yes in S103), that is, when the postures of all persons to be detected can be estimated from one image, one image is obtained for each person. The posture of the subject 11 is estimated from the image (here, the joint positions are estimated) (S108). Then, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 (S109).

　Steps S108 and S109 are performed as follows. First, the fatigue estimation system 200 acquires the image output by the imaging device 201 by the acquisition unit 101 . Here, when it is detected that the subject 11 is stationary (is in a stationary posture) in the acquired image, the estimation device 100 estimates the posture of the subject 11 . Specifically, first, the pressure distribution applied to the detection surface is obtained from the pressure sensor.

The posture estimation unit 105 estimates the posture of the subject 11 based on the acquired image and pressure distribution. The pressure distribution is used, for example, when biased pressure is applied, to correct the estimated pose to form that bias. Next, the first calculator 106 calculates the amount of load on each muscle and/or joint of the subject 11 from the posture estimation result. At this time, the personal information obtained in advance is used to correct and calculate the load amount. The estimation of the posture of the subject 11 is as described with reference to FIG. 1B, and the calculation of the amount of load is as described with reference to FIG. 1C, so detailed description thereof will be omitted.

In the correction of the load amount using personal information, for example, the closer the age of the subject 11 is to the peak age of muscle development, the smaller the load amount, and the farther from the peak age, the larger the load amount. Such peak values may be based on the subject's 11 gender. Also, if the sex of the subject 11 is male, the amount of load may be small, and if the sex of the subject 11 is female, the amount of load may be large. Alternatively, the smaller the height and weight of the subject 11, the smaller the load, and the larger the height and weight, the larger the load.

In addition, the load amount may be reduced as the composition ratio of the subject 11 has a large muscle mass, and the load amount may be increased as the composition ratio of the muscle mass of the subject 11 is small. Alternatively, the lower the stress level of the subject 11, the smaller the load amount, and the higher the stress level, the larger the load amount. Alternatively, the higher the body fat percentage of the subject 11, the larger the load amount, and the lower the body fat percentage, the smaller the load amount. Furthermore, the higher the proficiency level of the subject 11 with respect to the exercise, the lower the load amount, and the lower the proficiency level with respect to the exercise, the higher the load amount.

Here, the duration of the stationary posture of the subject 11 is measured based on the time acquired from the timing device. The fatigue estimating unit 108 adds the load amount calculated above each time the duration time passes by the unit time, and estimates the degree of fatigue of the subject 11 at this point. These processes are continued until the target person 11 is released from the stationary state. Specifically, whether or not the stationary state has been canceled is determined based on whether or not the orientation estimated by orientation estimation section 105 has changed from a certain static orientation.

If it is not determined that the stationary state has been released, the duration time is measured and the load amount is added, so that the fatigue level of the subject 11 is accumulated as long as the stationary posture continues. That is, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 using a fatigue level increasing function having a gradient corresponding to the calculated load amount with respect to the duration. Therefore, the greater the calculated load amount, the greater the degree of fatigue of the subject 11 that increases per unit time. In addition, in such accumulation of the fatigue level, the fatigue level of the subject 11 is initialized (set to 0) at the start timing of the stationary posture, which is the starting point.

On the other hand, when it is determined that the stationary state has been released, the posture estimation unit 105 calculates the amount of change in posture from the original stationary state posture to the current posture. The amount of change in posture is calculated for each individual muscle and/or joint, similar to the amount of load described above. When the posture changes in this way, the load on at least one of the muscles and joints changes, and in terms of the blood flow, the blood flow that has deteriorated is temporarily relieved, and the fatigue level of the subject 11 is recovered. turn into The degree of fatigue reduced by recovery is related to the amount of change in posture. Accordingly, the second calculation unit 107 calculates a recovery amount, which is the degree of recovery from fatigue, based on the amount of change in posture.

Based on the time acquired from the timing device, the change time, which is the time during which the change in posture of the subject 11 continues, is measured. The relationship between the recovery amount and the change time is the same as the relationship between the load amount and the duration time, and the recovery amount of the subject 11 is integrated as long as the posture change continues. In other words, the fatigue estimating unit 108 estimates the fatigue level of the subject 11 by subtracting the recovery amount each time the subject 11 passes the unit time at the timing when the posture of the subject 11 changes.

　The process of calculating the amount of recovery, measuring the change time, and estimating the degree of fatigue of the subject 11 continues until the posture of the subject 11 is stationary. Specifically, it is determined whether or not the posture estimated by posture estimation section 105 is a static posture. When the target person 11 is not detected to be stationary, the amount of recovery is calculated, the change time is measured, and the amount of recovery is subtracted so that the fatigue level of the target person 11 recovers as long as the change in posture continues. Accumulate.

That is, the fatigue estimation unit 108 estimates the fatigue level of the subject 11 using a fatigue level decreasing function having a slope corresponding to the calculated recovery amount with respect to the change time. Since the recovery amount of the fatigue level depends on the amount of change in posture, the greater the amount of change in posture, the greater the fatigue level of the subject 11, which decreases per unit time.

On the other hand, when the target person 11 is detected to be stationary, the posture and fatigue level are estimated again for a new static posture. In this way, in the fatigue estimation system 200, since the fatigue level of the subject 11 is calculated in consideration of the duration time in the stationary posture based on the image, the burden on the subject 11 is small, and more accurate The degree of fatigue of the subject 11 can be estimated.

The above will be explained in more detail with reference to FIGS. 4A to 5B. 4A is a diagram showing a subject standing still in Posture A. FIG. Moreover, FIG. 4B is a diagram showing the subject standing still in posture B. As shown in FIG.

A subject 11 shown in FIGS. 4A and 4B is in a stationary posture in a seated position on a chair 12, similar to that shown in FIG. 1A. In FIGS. 4A and 4B, there are actually tables, PCs, etc., which are not shown, but only the subject 11 and the chair 12 are shown here. The stationary posture of the subject 11 shown in FIG. 4A is posture A in which the load on the shoulders is relatively large. On the other hand, the stationary posture of the subject 11 shown in FIG. 4B is posture B in which the load on the shoulders is relatively small.

The degree of fatigue estimated for the subject 11 standing still in posture A or posture B is accumulated as shown in FIGS. 5A and 5B. FIG. 5A is a first diagram illustrating accumulation of estimated fatigue level of a subject according to the embodiment. Also, FIG. 5B is a second diagram for explaining the accumulation of the subject's fatigue level estimated according to the embodiment.

As shown in FIG. 5A, when the target person 11 is stationary in the posture A shown in FIG. 4A or the posture B shown in FIG. It is represented by a linear function with a slope.

As described above, posture A is a posture with a larger load than posture B. Therefore, for example, in certain muscles of the subject 11 (here, muscles related to shoulder movement), the load amount of posture A (slope of the straight line of posture A) is greater than the load of posture B (slope of the straight line of posture B). is also big. Therefore, in the posture A, the target person 11 accumulates (accumulates) a larger degree of fatigue in a shorter time than in the case of standing still in the posture B.

On the other hand, as shown in FIG. 5B, when the posture of subject 11 changes from posture A shown in FIG. 4A to posture B shown in FIG. is represented by a function in which a linear function whose gradient is and a linear function whose gradient is the amount of change in posture are connected.

Therefore, for example, in certain muscles of the subject 11, while the subject 11 is stationary in the posture A, the degree of fatigue of the subject 11 is a positive slope increasing function is estimated as the accumulation (addition) of the degree of fatigue by , and the accumulation (addition) turns to recovery (decrease) at the change point where the subject 11 begins to change the posture. The degree of fatigue of the subject 11 is determined by a decreasing function with a negative slope corresponding to the amount of change in posture. It recovers (decreases) by the amount shown as the width of change. After the change point at which the posture of the subject 11 is again stationary at the posture B, the fatigue level of the subject 11 is an increasing function with a positive slope corresponding to the load amount of the posture B, and is estimated as accumulation (addition) of the fatigue level. be done.

Thus, in the fatigue estimation system 200 of the present embodiment, the degree of fatigue of the subject 11 is estimated in which accumulation and recovery are reflected depending on whether the posture of the subject 11 is stationary or changed.

Returning to the description of the flowchart in FIG. 3, in step S103, when the fatigue estimation apparatus 100 determines that A and B do not match (No in S103), that is, one image in the person to be detected If a person whose posture cannot be estimated is included, such a person is identified (S104). Then, the posture estimation unit 105 selects and acquires a plurality of images that can be used for estimating the posture of the specified person (hereinafter referred to as a subject 11 whose fatigue level is estimated through synthesis of joint positions). (S105). Here, if the posture of the subject 11 cannot be estimated even using a plurality of images (No in S106), the process of estimating the degree of fatigue of the subject 11 ends. At this time, for example, notification information, which has been described as a function of the output unit 109, is output.

For the target person 11 whose posture can be estimated by using multiple images (Yes in S106), the posture of the target person 11 is estimated as shown in FIG. 6 below. FIG. 6 is a diagram illustrating an example of synthesizing joint positions according to the embodiment. FIG. 6 shows information generated during estimation of the posture (right end) of the subject 11 based on the acquired images (left end).

In FIG. 6, the image processing system is divided into upper and lower stages from the left side to the center. The joint positions 11c and 11c are shown. Similarly, in FIG. 6, another image 90b of the plurality of images and a joint position 11d in another body part estimated from the other image 90b are shown on the lower side. First, posture estimation section 105 acquires a plurality of images (one image 90a and another image 90b here) in step S105. The one image 90a and the other image 90b are images obtained by imaging one body part and another body part, which are part of the body part of the subject 11, respectively.

Also, one body part and another body part are body parts that share at least one joint with each other. Therefore, when a combination of a plurality of images sharing at least one joint with each other cannot be specified, the result of step S106 is No, and the process ends.

As shown in FIG. 3, the posture estimation unit 105 estimates the joint positions of the body parts of the subject 11 that are partially reflected in each of the plurality of images (S107). In the example of FIG. 6, based on one image 90a, a joint position 11c in one body part reflected in one image 90a is estimated, and first information 90c including this is generated. Similarly, in the example of FIG. 6, based on another image 90b, a joint position 11d of another body part reflected in another image 90b is estimated, and second information 90d including this is generated. After that, the joint positions 11c of the one body part and the joint positions 11d of the other body parts are combined to estimate the joint positions 11a of the body parts of the subject 11 including the one body part and the other body parts. do. In this manner, the posture estimation unit 105 can estimate the posture of the subject 11 using a plurality of images in which a part of the body part of the subject 11 is captured.

Here, the synthesizing of joint positions will be described while continuing to refer to FIG. Synthesis of the joint positions of the subject 11 requires two or more common coordinates between the joint position 11c in at least one body part and the joint position 11d in another body part. One of the common coordinates is the joint position of a shared joint, which is one joint shared between one body part and another body part. By using the reference coordinates shared in the plurality of images in addition to the joint positions of the shared joints, the scales of the plurality of images and the poses (orientations) of the body parts in the images can be matched.

Among the joints shared between one body part and another body part, the joint positions of the joints that are different from the shared joints are used as the reference coordinates. In this example, posture estimating section 105, that is, in a situation in which two or more joints are shared between one body part and another body part, the posture estimating unit 105 calculates the distance between these joints from one joint to the other joint. The joint position 11c in one body part and the joint position 11d in the other body part are adjusted so that the directions and magnitudes of the vectors extending to the body parts match. Then, the posture estimating unit 105 superimposes the joint position 11c of one body part and the joint position 11d of another body part with the shared joint as a reference while the posture (orientation) and scale match. By synthesizing, the joint positions 11a of the body parts of the subject 11 are estimated.

In addition to the above, markers appearing in both one image 90a and another image 90b may be used as the reference coordinates. In the example of FIG. 6, both one image 90a and another image 90b show a commonly moving object marker M. In the example of FIG. The object marker M may be a dedicated object, or some object that exists in space and can be identified as the same object by multiple imaging devices 201 may be used. If the spatial coordinates of the object marker M are used as the reference coordinates, the posture (orientation) and scale can be matched, and the shared joints can be used as a reference to superimpose and combine. For this purpose, the posture estimation unit 105 may calculate the spatial coordinates Mc of the object marker M from one image 90a, and the spatial coordinates Md of the object marker M from the other image 90b.

A line marker L may also be used as another example of the marker. For example, the line marker L is realized by attaching a tape or the like to any location in the space. The line marker L has both ends in common and is configured to extend between the two ends. The same effect as above can be obtained even if it does not exist.

Further effects can be obtained by using the fact that these markers exist in the real space. That is, the fatigue estimation system 200 can distinguish the target person 11 from other persons staying in the same space as the target person 11 based on the positional relationship between the marker and the target person 11 . In the present embodiment, processing such as superimposing the joint positions for each body part of the subject 11 is performed from a plurality of images. Therefore, when another person (for example, another person 11z in FIG. 6) is staying in the same space as the subject 11, the joint position of one body part of the subject 11 and the other person's other If the joint positions of the body parts are superimposed, the posture of the subject 11 cannot be estimated normally.

Therefore, in the fatigue estimation system 200, the identification unit 102 is provided to identify the target person 11 and reliably overlap the joint positions of a part of the target person 11 with the joint positions of the other part of the target person 11. can. For this reason, the identification unit 102 detects the relative positions of the subject 11 and the marker (the subject 11 is positioned at a certain distance in a certain direction with respect to the marker) in the image acquired from the imaging device 201 . Recognizing that another person 11z is staying at a position advanced by a different distance in the direction This process may be performed with the spatial resolution of one person, so the process of estimating each joint position It has less processing load than , and can be easily implemented.

As described above, in the present embodiment, even an image showing only a part of the body part of the subject 11 can be used for estimating the degree of fatigue by combining a plurality of images to complement each other. posture can be estimated.

Further, as described above, when estimating the joint positions of the body parts including a part of the body part of the subject 11 and another part of the body part as the posture of the subject 11, the body part of the subject 11 There are cases where posture estimation cannot be performed for other parts that are not included in a part of and another part. For example, the posture of an invisible part that is not captured in any of the plurality of images, that is, is not captured in any of the images, cannot be estimated by the above processing.

In that case, for example, by inputting the joint positions of a part of the body part of the subject 11 estimated by the above and another part into a machine learning model in which the joint positions of the whole body of the person are learned, may be estimated as the posture of the whole body of the subject 11 . Alternatively, the posture of the subject 11 may be estimated by adding only a part of the invisible parts that has sufficient reliability as an output in the machine learning model, instead of the whole body. In this manner, the posture estimation unit 105 is configured to estimate the joint positions of the body parts of the subject 11 including invisible parts that are not captured in a plurality of images as the posture of the subject 11. good too.

Returning to FIG. 3 again, as described above, the posture estimation unit 105 estimates the posture of the subject 11 through processing such as synthesis using a plurality of images in which a part of the body part of the subject 11 is captured. (S108). Then, in step S109, the fatigue estimating unit 108 estimates the degree of fatigue of the subject 11 as described above.

Next, an example of the output of the output unit 109 based on the estimated degree of fatigue will be described. FIG. 7 is a diagram showing a display example of estimation results according to the embodiment.

As shown in FIG. 7, in the fatigue estimation system 200, the result of estimating the degree of fatigue of the subject 11 can be displayed using the display device 205 and fed back. Specifically, as shown in FIG. 7, by visualizing the degree of fatigue of the subject 11, it is possible to visually grasp how tired the subject 11 is. In the figure, a display device 205 integrally displays a doll simulating the subject 11 and fatigue levels of the subject 11 in the shoulders, back, and waist. In order to make it easier for the subject 11 to intuitively grasp the fatigue level, the fatigue level of the shoulder is indicated as "stiff shoulder", the fatigue of the back is indicated as "back pain", and the fatigue of the lower back is indicated as "low back pain". It is

Here, in the display in the figure, the fatigue levels of the subject 11 at three locations are displayed at once, and the fatigue levels at these three locations are estimated from images captured at once. That is, the estimating apparatus 100 detects the muscles and/or joints in each of a plurality of body parts including the first part (eg, shoulder), the second part (eg, back), and the third part (eg, waist) of the subject 11. , the degree of fatigue is estimated from one posture of the subject 11 . Therefore, even if the posture of the subject 11 is constant, the degree of fatigue accumulated in muscles and/or joints differs for each body part. can be estimated.

As described with reference to FIG. 1C, in the present embodiment, the load amount is calculated for each muscle and/or joint of the subject 11. Therefore, if there is no processing resource limitation, each muscle and/or joint fatigue can be estimated. Therefore, there is no limit to the number of body parts whose fatigue levels are estimated from images captured at one time, and the number may be one, two, or four or more.

The estimating device 100 calculates the load amount for each of a plurality of body parts, and calculates the degree of fatigue (the degree of stiff shoulders) of the first part based on the load amount calculated for the first part in one posture of the subject 11. , the fatigue level of the second part (the above-mentioned back pain level) due to the load amount calculated at the second part, and the fatigue level of the third part (the above-mentioned low back pain level) due to the load amount calculated at the third part, can be estimated.

In the example in the figure, the degree of stiff shoulders is estimated from the amount of load on the trapezius muscle, the degree of back pain is estimated from the degree of fatigue of the latissimus dorsi muscle, and the degree of low back pain is estimated from the amount of load on the lumbar paraspinal muscles. In this way, one fatigue level may be estimated from the load of one muscle and / or joint, but one fatigue level is estimated from the combined load of a plurality of muscles and / or joints. may For example, the degree of stiff neck (that is, one degree of shoulder fatigue) may be estimated from the average value of the loads of the trapezius muscle, the levator scapula muscle, the rhomboid muscle, and the deltoid muscle. In addition, in estimating the degree of fatigue, rather than a simple average value, by weighting the load amount of muscles and / or joints that have a particularly large effect on the degree of fatigue of the body part according to the magnitude of the effect, A more realistic estimation of the degree of fatigue may be performed.

The degree of fatigue estimated in this way may be indicated as a relative position on a reference meter with a minimum value of 0 and a maximum value of 100 as shown. Here, a reference value is provided at a predetermined position on the reference meter. Such a reference value is set to the relative position (or before and after) of the degree of fatigue at which subjective symptoms such as pain may occur in a general subject 11 quantified in advance by an epidemiological survey or the like. Therefore, different reference values may be set according to the degree of fatigue of each body part.

Furthermore, the display device 205 may display a warning to the target person 11 as an estimation result when the estimated fatigue level of the target person 11 reaches the reference value. The reference value here is an example of the first threshold. In the drawing, as an example of such a warning, "the degree of stiff shoulders exceeds the reference value" is displayed at the bottom of the display device 205. Further, in relation to such a warning, the display device 205 may display a specific coping method such as "Recommend taking a break" as shown in the drawing.

As described above, in addition to the configuration that prompts the subject 11 to cope with the accumulated fatigue level by displaying the estimation result to the subject 11, the fatigue estimation system 200 actively activates the subject 11. It is also possible to consider a configuration that recovers the degree of fatigue of the user. Specifically, the recovery device 206 shown in FIG. 2 operates to recover the fatigue level of the subject 11 . The specific configuration of the recovery device 206 is as described above, so a description thereof will be omitted. By changing the load on at least one of the muscles and joints of the subject 11 and promoting blood circulation, the subject's degree of fatigue is reduced. The reference value here is an example of the third threshold, and may be the same as or different from either the first threshold or the second threshold.

[Effects, etc.]
As described above, the fatigue estimation system 200 in the present embodiment includes a plurality of imaging devices 201 each outputting an image in which a different part of the body part of the subject 11 is captured, and a plurality of imaging devices A posture estimation device (posture estimation unit 105) for estimating the posture of the subject 11 based on the plurality of images output in each of the 201, and a fatigue estimating device 100 that estimates and outputs the degree.

In such a fatigue estimation system 200, the posture estimation unit 105 cannot obtain an image showing the whole body of the subject 11 in one image due to conditions such as the arrangement position of the imaging device 201. In a situation where it is difficult to estimate the posture of the subject 11, the posture of the subject 11 can be appropriately estimated by using a combination of different images. Then, it is possible to more appropriately estimate the fatigue level of the subject 11 based on the posture that is appropriately estimated by the posture estimation unit 105 .

Further, for example, the posture estimation unit 105 estimates the joint position 11c in one body part based on one image 90a in which one body part is captured among the plurality of images, and , based on another image 90b in which another body part sharing at least one joint with the one body part is imaged, the joint position 11d in the other body part is estimated, and the estimated joint position in the one body part 11 c and the estimated joint positions 11 d of other body parts may be combined to estimate joint positions 11 a of body parts including one body part and other body parts as the posture of the subject 11 .

According to this, the joint position 11a of the body part is estimated using one image 90a in which one body part is imaged and another image 90b in which the other body part is imaged, among the plurality of images. can do. Here, a joint position 11c in one body part is estimated from one image 90a, a joint position 11d in another body part is estimated from another image 90b, and joint positions 11d in another body part are estimated, and these are combined to obtain the one body part and the other body part. can estimate the joint positions 11a of the body parts including the body parts of In this way, it is possible to estimate joint positions 11a that include both other body parts that are not included in one image 90a and one body parts that are not included in another image 90b.

Further, for example, in synthesizing the estimated joint position 11c in one body part and the estimated joint position 11d in another body part, a shared joint that is shared between one body part and another body part Based on the relative positions of the joints and the reference coordinates shared in the plurality of images, at least one of the relative pose and the relative scale of the joint position 11c in one body part and the joint position 11d in the other body part is determined. The shared joint and the reference coordinates are superimposed with at least one of the relative posture and the relative scale between the determined joint position 11c in one body part and the determined joint position 11d in another body part being applied. may

According to this, by superimposing the relative positions of the shared joints and the reference coordinates, the joint position 11c in one body part is obtained from one image 90a, and the joint position 11d in another body part is obtained from another image 90b. can be synthesized. At this time, depending on the arrangement position of the imaging device 201, at least one of the posture and the scale of the joint position 11c in one body part and the joint position 11d in the other body part differs between the two images. sell. In the above aspect, the relative posture (for example, the amount of rotation to eliminate the difference in relative orientation) and the relative scale (for example, A scaling ratio for canceling a relative scale difference) can be applied to adjust different poses, scales, etc. between the two images, and then the above synthesis can be performed.

Also, for example, the reference coordinates may be the joint position of a joint different from the shared joint.

According to this, the joint position of the joints different from the shared joint is used as the reference coordinates and the relative position between the shared joint and the reference coordinates is overlaid, so that the joint position 11c in one image 90a is 11c, And joint positions 11d in other body parts can be synthesized from other images 90b.

Also, for example, the reference coordinates may be spatial coordinates of a marker placed at a position captured in any of the plurality of images.

According to this, by using the spatial coordinates of the marker arranged at the position captured in any of the plurality of images as the reference coordinates and superimposing the relative positions of the shared joints and the reference coordinates, one image 90a can be obtained. The joint position 11c in one body part can be synthesized from the image 90b, and the joint position 11d in another body part can be synthesized from the other image 90b.

Further, for example, an identification unit 102 may be provided that identifies the target person 11 from another person 11z staying in the same space as the target person 11 based on the positional relationship between the marker and the target person 11.

According to this, based on the positional relationship between the marker and the target person 11, the target person 11 is identified from other persons 11z staying in the same space as the target person 11, and the posture of only the target person 11 is estimated. be able to. As a result, the fatigue level can be estimated only for the target person 11 even when another person 11z is present.

Further, for example, when the posture estimation unit 105 fails to estimate the posture of the target person 11, the posture estimation unit 105 outputs an estimation result indicating that the estimation of the posture of the target person 11 has failed, and the fatigue estimation system 200 further: An output unit 109 may be provided that outputs notification information for notifying the presence of the target person 11 whose posture estimation has failed, based on the estimation result indicating that the posture estimation of the target person 11 has failed.

According to this, it is possible to know the presence of the target person 11 whose posture estimation has failed based on the notified notification information.

Further, for example, the posture estimating unit 105 estimates the joint positions of the partial body parts of the subject 11 based on a plurality of images, and converts the estimated joint positions of the partial body parts of the subject 11 into human body parts. The joint positions of the whole body are input to a learned machine learning model, and the joint positions 11a of the body parts of the subject 11 including invisible parts that are not captured in a plurality of images are estimated as the posture of the subject 11. You may

According to this, by using a machine learning model in which the joint positions of the whole human body are learned, the joint positions 11a can be estimated for the body parts including the invisible parts of the subject 11 that cannot be estimated from any combination of a plurality of images. can do

Further, for example, the posture estimation unit 105 further acquires the number of detection target persons in the area, and among the detection target persons, only one image output from one imaging device 201 that captures an image of the area is used. Calculate the number of people whose postures can be estimated, and if the number of people whose postures can be estimated differs from the number of people whose postures can be estimated, among the people who are not postures can be estimated. 11, the posture of the subject 11 may be estimated.

According to this, when the number of detection targets in an area is different from the number of persons whose postures can be estimated, there is a target 11 among the detection targets whose posture cannot be estimated unless a plurality of images are combined. The above pose estimation using a plurality of images can be performed only when Therefore, processing resources for estimating the posture can be reduced, and the fatigue estimation system 200 can be realized with a simple configuration.

Also, the posture estimation device according to the present embodiment is posture estimation section 105 described above.

According to this, an image showing the entire body of the subject 11 cannot be obtained in one image depending on conditions such as the arrangement position of the imaging device 201, and it is difficult to estimate the posture of the subject 11 from one image. In some situations, a combination of different images can be used to better estimate the pose of the subject 11 .

Further, the fatigue estimation method in the present embodiment is a fatigue estimation method executed by the fatigue estimation apparatus 100, and includes a plurality of images each outputting an image in which a different part of the body part of the subject 11 is captured. , and share one image 90a in which one body part among the plurality of images is captured, and one body part and at least one joint among the plurality of images. and another image 90b in which another body part is imaged, and joint positions 11a of body parts including one body part and another body part are estimated as the posture of the subject 11, and the subject 11 Based on the posture estimation result, the fatigue level of the subject 11 is estimated and output to an output device connected to the fatigue estimation apparatus 100 .

According to this, the same effect as the fatigue estimation system 200 described above can be obtained.

Also, the program in the present embodiment is a program for causing a computer to execute the fatigue estimation method described above.

According to this, it is possible to obtain the same effect as the fatigue estimation system 200 described above using a computer.

(Other embodiments)
Although the embodiments have been described above, the present disclosure is not limited to the above embodiments.

For example, in the above embodiment, the processing executed by a specific processing unit may be executed by another processing unit. In addition, the order of multiple processes may be changed, and multiple processes may be executed in parallel.

In addition, the fatigue estimation system or posture estimation device in the present disclosure may be implemented with multiple devices each having a portion of multiple components, or may be implemented with a single device having all of the multiple components. good too. Also, some of the functions of a component may be implemented as functions of another component, and each function may be distributed among the components in any way. The present disclosure includes any form having a configuration that includes substantially all of the functions that can realize the fatigue estimation system or the posture estimation device of the present disclosure.

Also, in the above embodiments, each component may be realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Also, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form one circuit as a whole, or may be separate circuits. These circuits may be general-purpose circuits or dedicated circuits.

Also, general or specific aspects of the present disclosure may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM. Also, any combination of systems, devices, methods, integrated circuits, computer programs and recording media may be implemented.

Further, in the above embodiment, the posture of the subject is estimated from the image using the rigid link model generated by image recognition, the load amount is calculated from the estimated posture of the subject, and based on the load amount and the duration time. However, the method of estimating the degree of fatigue is not limited to this. Any existing method may be used as a method of estimating the posture of the subject from the image, and any existing method may be used as a method of estimating the amount of load from the posture of the subject.

Also, in the above embodiment, the increasing function and decreasing function are described as linear functions, but the present invention is not limited to this. The increasing function may be a curvilinear function as long as the fatigue level increases with time. Also, the decreasing function may be a curvilinear function as long as it is a function that decreases the degree of fatigue over time.

In addition, the estimation device described above uses the load on muscles, the load on joints, and the blood flow estimated from the posture of the subject to estimate the degree of fatigue of the subject. As described above, it is also possible to correct the estimated value using the value measured using the measuring device to achieve more accurate estimation of the degree of fatigue. Specifically, the estimating device acquires a measured value corresponding to the estimated value, which is a measured value based on the measurement result of measuring the subject by the measuring device.

The detection device is, for example, an electromyograph, a muscle hardness meter, a pressure gauge, a near-infrared spectrometer, etc., and obtains measurement values regarding the amount of load on muscles, the amount of load on joints, and the amount of blood flow. can be done. For example, an electromyograph can estimate muscle movement corresponding to the potential based on the potential measured by potential measurement. In other words, a value obtained by estimating the muscle movement can be obtained as a measured value. Since the value obtained by estimating the movement of the muscle can be converted into the amount of load on the muscle, the estimated value of the amount of load on the muscle can be corrected by the measured value. The correction here is, for example, taking the average value of the estimated value and the measured value, selecting one of the estimated value and the measured value, and applying the estimated value to the correlation function between the estimated value and the measured value. and so on.

A muscle hardness meter can estimate muscle hardness from the stress when pressure is applied to the muscle. Since the estimated muscle hardness value can be converted into the amount of load on the muscle, it can be used to correct the estimated value in the same manner as described above.

The pressure gauge can obtain a measured value of what kind of pressure is applied to the body part of the subject. Such pressure parameters can be input into the musculoskeletal model. By inputting additional parameters such as pressure, the estimation accuracy of the musculoskeletal model is improved, and the estimated value estimated using the musculoskeletal model can be corrected with higher accuracy.

A near-infrared spectrometer can obtain spectroscopic measurement values of the subject's blood flow. When the blood flow rate is not included in the estimated value as in the above embodiment, the estimated value may be corrected by combining the blood flow rate measured by the infrared spectrometer. Moreover, even if the estimated value includes the blood flow, the measured blood flow may be used when the estimated blood flow has low reliability.

In this way, by using the measured values corresponding to the estimated values obtained from different aspects and correcting the estimated values to make them more accurate, more accurate estimation of the fatigue level of the subject can be performed. becomes possible.

In addition, the fatigue estimation system described in the above embodiment may be used to configure a fatigue factor identification system that identifies the subject's fatigue factors. Conventional devices or systems for estimating the degree of fatigue as "degree of stiff shoulder" and "degree of low back pain" use muscles and joints (that is, factors It was difficult to identify the posture that Therefore, by using the fatigue estimation system according to the present disclosure, the above problem can be addressed.

That is, in the fatigue factor identification system of the present disclosure, body parts where fatigue is likely to accumulate (body parts with large estimated amounts that promote various types of fatigue) are identified as fatigue factor parts in the static posture taken by the subject. Furthermore, the fatigue factor identification system may simply identify the fatigue factor part in one static posture taken by the subject, and the estimated amount in the fatigue factor part most among the plurality of static postures taken by the subject You may also identify the fatigue factor posture with many In addition, a recommended posture that replaces the specified fatigue-causing posture may be presented, and a fatigue degree recovery operation using a recovery device may be performed on the fatigue-causing portion in the fatigue-causing posture.

The fatigue factor identification system includes the fatigue estimation system described in the above embodiment and a storage device for storing information on the estimated degree of fatigue. Such a storage device may be implemented using, for example, a semiconductor memory or the like, and each main storage unit or the like constituting the fatigue estimation system may be used. good too.

Also, the present disclosure may be implemented as a fatigue estimation method executed by a fatigue estimation system or an estimation device. The present disclosure may be implemented as a program for causing a computer to execute such a fatigue estimation method, or may be implemented as a computer-readable non-temporary recording medium in which such a program is recorded. .

In addition, a form obtained by applying various modifications to the embodiment that a person skilled in the art can think of, or realized by arbitrarily combining the components and functions in each embodiment within the scope of the present disclosure Forms are also included in this disclosure.

11 Subject 11a Joint position 11c Joint position in one body part 11d Joint position in another body part 11z Other person 90a One image 90b Another image 100 Fatigue estimation device 101 Acquisition unit 102 Identification unit 105 Posture estimation unit (Posture estimation device)
109 Output unit 200 Fatigue estimation system 201 Imaging device Mc, Md Spatial coordinates

Claims (12)

1. a plurality of imaging devices each outputting an image in which a different part of a subject's body part is captured;
   a posture estimation device that estimates the posture of the subject based on the plurality of images output from each of the plurality of imaging devices;
   A fatigue estimation system, comprising: a fatigue estimation device that estimates and outputs a degree of fatigue of the subject based on estimation results of the posture of the subject.
2. The posture estimation device is
   estimating a joint position in the one body part based on one of the plurality of images in which one body part is captured;
   estimating a joint position in the other body part based on another image among the plurality of images in which another body part sharing at least one joint with the one body part is imaged;
   By synthesizing the estimated joint positions of the one body part and the estimated joint positions of the other body parts, the joint positions of the body parts including the one body part and the other body parts are obtained by the object. The fatigue estimation system according to claim 1, wherein the fatigue estimation system estimates the posture of a person.
3. In synthesizing the estimated joint position in the one body part and the estimated joint position in the other body part,
   joints in the one body part based on relative positions of shared joints that are joints shared between the one body part and the other body part and reference coordinates shared in the plurality of images; determining at least one of a relative pose and a relative scale of a position and a joint position in said other body part;
   superimposing the shared joints and the reference coordinates in a state in which at least one of a relative posture and a relative scale between the determined joint positions of the one body part and the joint positions of the other body parts is applied; Item 3. The fatigue estimation system according to item 2.
4. The fatigue estimation system according to claim 3, wherein the reference coordinate is a joint position of a joint different from the shared joint.
5. The fatigue estimation system according to claim 3, wherein the reference coordinates are spatial coordinates of a marker arranged at a position captured in any of the plurality of images.
6. The fatigue estimation system according to claim 5, further comprising an identification unit that identifies the target person from other persons staying in the same space as the target person based on the positional relationship between the marker and the target person.
7. The posture estimation device outputs the estimation result indicating that the estimation of the posture of the subject is unsuccessful when estimation of the posture of the subject is unsuccessful;
   The fatigue estimation system further outputs notification information for notifying the presence of the subject whose posture estimation has failed, based on the estimation result indicating that the subject's posture has failed to be estimated. The fatigue estimation system according to any one of claims 1 to 6, comprising a unit.
8. The posture estimation device is
   estimating joint positions in a part of the subject's body based on the plurality of images;
   inputting the estimated joint positions in the partial body part of the subject into a machine learning model in which the joint positions of the whole human body have been learned;
   Joint positions of body parts including invisible parts that are not imaged in a plurality of the images of the body parts of the subject, to estimate as the posture of the subject Fatigue according to any one of claims 1 to 7 estimation system.
9. The posture estimation device further comprises:
   Get the number of people to be detected in the area,
   Calculating the number of people whose posture can be estimated from only one image output from one imaging device that captures an image of the area, among the detection target people,
   When the number of the detection target persons and the number of the posture estimable persons are different, estimating the posture of the target person among the detection target persons who are not the posture estimable persons as the target person. Item 9. The fatigue estimation system according to any one of items 1 to 8.
10. The posture estimation device according to any one of claims 1 to 9.
11. A fatigue estimation method performed by a fatigue estimation device,
    Acquiring the images from each of a plurality of imaging devices that each output an image in which a different part of the subject's body part is captured;
    one image obtained by imaging one body part among the plurality of images, and another body part sharing at least one joint with the one body part among the plurality of images obtained by imaging and estimating the joint positions of the body parts including the one body part and the other body part as the posture of the subject, based on the image of
    A fatigue estimation method, comprising: estimating a degree of fatigue of the subject based on an estimation result of the posture of the subject, and outputting the fatigue level to an output device connected to the fatigue estimation device.
12. A program for causing a computer to execute the fatigue estimation method according to claim 11.

Images