WO2022209287A1 - Calculation device - Google Patents

Abstract

A calculation device 800 has: a height calculation unit 821 that calculates, on the basis of image data, a height of a person in the image data, from the head to the tips of feet; and a bending calculation unit 822 for acquiring a hand-to-hand length that is the skeleton length between left and right hands of the person in the image data, and for calculating an index value indicating bending of the waist of the person in the image data on the basis of the acquired hand-to-hand length and the height calculated by the height calculation unit 821.

Description

calculator

The present invention relates to a calculation device, a calculation method, and a recording medium.

It is known to perform user gait analysis based on data.

For example, Patent Document 1 is a document describing human gait analysis. In Patent Document 1, a data acquisition unit that acquires two types of image data from a depth sensor, a skeleton information creation unit that creates skeleton information based on the image data acquired by the data acquisition unit, and a skeleton information creation unit that creates A gait analysis device having a correction processing unit that corrects skeleton information and an analysis processing unit that analyzes a user's gait using the corrected skeleton information is described.

WO2017/170832

An example of an index that you would like to calculate when performing gait analysis is the curvature of the waist. However, for example, when performing an analysis based on image data without using a depth sensor as described in Patent Document 1, since information in the depth direction cannot be obtained, an index value corresponding to the bending of the waist cannot be obtained. was difficult to calculate.

Therefore, an object of the present invention is to provide a determination device, a determination method, and a recording medium that solve the problem that it is difficult to calculate an index value corresponding to the curvature of the waist based on image data.

In order to achieve such an object, a computing device, which is one aspect of the present disclosure,
a height calculation unit that calculates the height of a person in the image data based on the image data;
A hand length, which is a length connecting the left and right hands of a person in the image data, is obtained, and based on the obtained hand length and the height calculated by the height calculation unit, a curvature calculation unit that calculates an index value indicating the curvature of the waist of the person;
It has a configuration of

In addition, a calculation method, which is another aspect of the present disclosure,
The information processing device
Based on the image data, calculate the height of the person in the image data,
A hand length, which is a length connecting the left and right hands of the person in the image data, is obtained, and the curve of the waist of the person in the image data is calculated based on the obtained hand length and the calculated height. Calculating an index value indicating

In addition, a recording medium that is another aspect of the present disclosure includes:
information processing equipment,
Based on the image data, calculate the height of the person in the image data,
A hand length, which is a length connecting the left and right hands of the person in the image data, is obtained, and the curve of the waist of the person in the image data is calculated based on the obtained hand length and the calculated height. It is a computer-readable recording medium in which a program for realizing a process of calculating an index value indicating is recorded.

According to each configuration as described above, it is possible to calculate an index value corresponding to the bending of the waist based on the image data.

1 is a diagram illustrating a configuration example of a determination system according to a first embodiment of the present disclosure; FIG. FIG. 4 is a diagram for explaining an example of image data acquired by the determination system; FIG. 2 is a block diagram showing a configuration example of a determination device shown in FIG. 1; FIG. FIG. 4 is a diagram showing an example of skeleton information; It is a figure which shows an example of grounding determination information. FIG. 10 is a diagram for explaining an example of determination by a contact determination unit; FIG. 4 is a diagram for explaining an example of determination by a touch timing determination unit; 5 is a flow chart showing an example of processing by a ground contact determination unit; 6 is a flow chart showing an example of processing by a contact timing determination unit; FIG. 11 is a diagram illustrating a configuration example of a calculation system according to a second embodiment of the present disclosure; FIG. 11 is a block diagram showing a configuration example of the calculation device shown in FIG. 10; FIG. It is a figure which shows an example of hand length and height. It is a flowchart which shows the operation example of a calculation apparatus. FIG. 11 is a diagram illustrating a configuration example of a calculation system according to a third embodiment of the present disclosure; FIG. 15 is a block diagram showing a configuration example of the calculation device shown in FIG. 14; FIG. It is a figure which shows an example of height. It is a figure which shows an example of a component. It is a figure which shows another example of a component. It is a flowchart which shows the operation example of a calculation apparatus. FIG. 13 is a diagram illustrating a hardware configuration example of a computing device according to the fourth embodiment of the present disclosure; FIG. It is a block diagram which shows the structural example of a calculation apparatus. FIG. 11 is a block diagram showing another configuration example of the calculation device;

[First embodiment]
A first embodiment of the present disclosure will be described with reference to FIGS. 1 to 9. FIG. FIG. 1 is a diagram showing a configuration example of a determination system 100. As shown in FIG. FIG. 2 is a diagram for explaining an example of image data acquired by the determination system 100. As shown in FIG. FIG. 3 is a block diagram showing a configuration example of the determination device 300. As shown in FIG. FIG. 4 is a diagram showing an example of the skeleton information 323. As shown in FIG. FIG. 5 is a diagram showing an example of the contact determination information 324. As shown in FIG. FIG. 6 is a diagram for explaining an example of determination by the contact determination unit 333. In FIG. FIG. 7 is a diagram for explaining an example of determination by the touchdown timing determining section 334. In FIG. FIG. 8 is a flow chart showing an example of processing by the contact determination unit 333. As shown in FIG. FIG. 9 is a flow chart showing an example of processing by the contact timing determination section 334. As shown in FIG.

In the first embodiment of the present disclosure, a determination system 100 that determines the timing at which the foot of a walking person touches the ground based on image data acquired using an imaging device such as the smartphone 200 will be described. The determination system 100 acquires time-series image data showing how a person walks from the back side to the front side of the screen. As will be described later, the determination system 100 recognizes the position of the skeleton from each of the plurality of image data. Then, the determination system 100 determines the timing at which the foot touches the ground based on the recognized change in position.

It should be noted that the timing of landing determined by the determination system 100 can be, for example, an index such as the angle of each joint at the timing when the foot touches the floor, whether the toe is raised from the floor, pitch, stride, various steps such as the step cycle. It can be used for gait analysis. The touchdown timing determined by the determination system 100 may be used in addition to the above examples.

FIG. 1 shows a configuration example of the determination system 100. FIG. Referring to FIG. 1 , the determination system 100 has, for example, a smart phone 200 and a determination device 300 . As shown in FIG. 1, the smartphone 200 and the determination device 300 are connected, for example, wirelessly or by wire so that they can communicate with each other.

The smart phone 200 functions as an imaging device that captures how a person walks. The smartphone 200 may be a smartphone having general functions such as a camera function for acquiring image data, a touch panel for displaying a screen, and various sensors such as a GPS sensor and an acceleration sensor.

In the case of this embodiment, as shown in FIG. 2, the smartphone 200 shoots a person walking in the front-back direction, such as from the back of the screen to the front. In other words, smartphone 200 acquires time-series image data showing how a person walks in the front-to-back direction. The smartphone 200 also transmits the acquired image data to the determination device 300 . The smartphone 200 may associate information indicating the date and time when the image data was acquired with the image data, and transmit the associated data to the determination device 300 .

The determination device 300 is an information processing device that determines the timing at which the foot of a walking person touches the ground based on the image data acquired by the smartphone 200 . For example, the determination device 300 is a server device or the like. The determination device 300 may be a single information processing device, or may be implemented on a cloud, for example.

FIG. 3 shows a configuration example of the determination device 300. FIG. Referring to FIG. 3, the determination device 300 has, for example, a communication I/F section 310, a storage section 320, and an arithmetic processing section 330 as main components.

The communication I/F unit 310 consists of a data communication circuit. Communication I/F unit 310 performs data communication with an external device, smartphone 200, or the like connected via a communication line.

The storage unit 320 is a storage device such as a hard disk or memory. The storage unit 320 stores processing information and programs 326 necessary for various processes in the arithmetic processing unit 330 . The program 326 realizes various processing units by being read and executed by the arithmetic processing unit 330 . The program 326 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 310 and stored in the storage unit 320 . Main information stored in the storage unit 320 includes, for example, a learned model 321, image information 322, skeleton information 323, contact determination information 324, contact timing information 325, and the like.

The trained model 321 is a trained model used when the skeleton recognition unit 332 performs skeleton recognition. The learned model 321 is, for example, generated in advance by performing machine learning using teacher data such as image data containing skeletal coordinates in an external device, etc. It is acquired from a device or the like and stored in the storage unit 320 . Trained model 321 may be generated using known methods. The learned model 321 may be updated by re-learning processing using additional teacher data.

The image information 322 includes time-series image data acquired by the camera of the smartphone 200 . The image information 322 is generated and updated when the image acquisition unit 331 acquires image data.

For example, in the image information 322, identification information for identifying image data is associated with image data. The image information 322 may include information other than the above examples, such as information indicating the date and time when the smartphone 200 acquired the image data.

The skeleton information 323 includes information indicating the coordinates of each part of the person recognized by the skeleton recognition unit 332 . The skeleton information 323 includes, for example, information indicating the coordinates of each part of each piece of time-series image data. For example, the skeleton information 323 is generated/updated as a result of processing by the skeleton recognition unit 332 .

FIG. 4 shows an example of the skeleton information 323. Referring to FIG. 4, in skeleton information 323, for example, identification information is associated with position information of each part. Here, the identification information is, for example, information indicating image data used when recognizing skeleton information. The identification information may correspond to identification information for identifying image data in the image information 322 . Also, the position information of each part includes information indicating the coordinates of each part in the image data, such as the position of the pelvis.

It should be noted that the part included in the position information of each part corresponds to the learned model 321 . For example, in FIG. 4, the pelvis, the center of the spine, ..., the right knee, the left knee, ..., the right ankle, the left ankle, ...
is exemplified. The position information of each part can include about 30 parts such as right shoulder, . . . , left elbow, . In the case of this embodiment, the position information of each part includes at least information indicating the coordinates of the right ankle and the left ankle. The parts included in the position information of each part may be other than those illustrated in FIG. 4 and the like.

The ground contact determination information 324 includes information indicating the determination result of the ground contact determination unit 333 . For example, the ground contact determination information 324 is generated and updated as a result of determination by the ground contact determination section 333 .

FIG. 5 shows an example of the contact determination information 324. FIG. As shown in FIG. 5, for example, in contact determination information 324, identification information and contact determination results are associated with each other. Here, the identification information is, for example, information indicating image data for which grounding determination has been performed. The identification information may correspond to identification information for identifying image data in the image information 322 . Further, the ground contact determination result indicates the ground contact determination result. For example, the ground contact determination result is "left foot" indicating the result of determining that the left foot is on the ground, "right foot" indicating the result of determining that the right foot is on the ground, and "determining that it is excluded from the ground contact determination". -" etc. are included. The contact determination result may indicate other than the above examples.

The touchdown timing information 325 includes information indicating the determination result of the touchdown timing determination unit 334 . For example, the touchdown timing information 325 is generated and updated as a result of determination by the touchdown timing determination section 334 .

For example, the ground contact timing information 325 includes information indicating the timing at which the foot touches the ground. Here, the information indicating the timing at which the foot touched the ground is, for example, at least one of identification information indicating the image data determined to have touched the ground, information indicating the acquisition date and time of the image data determined to have touched the ground, and the like. be. The contact timing information 325 may be, for example, information indicating the foot that has touched the ground and information indicating the timing at which the foot touched the ground in association with each other.

The arithmetic processing unit 330 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 330 reads the program 326 from the storage unit 320 and executes it, so that the hardware and the program 326 work together to realize various processing units. Main processing units realized by the arithmetic processing unit 330 include, for example, an image acquisition unit 331, a skeleton recognition unit 332, a contact determination unit 333, a contact timing determination unit 334, an output unit 335, and the like. .

The image acquisition unit 331 acquires image data acquired by the smartphone 200 from the smartphone 200 via the communication I/F unit 310 . For example, the image acquisition unit 331 acquires time-series image data from the smartphone 200 . The image acquisition unit 331 also stores the acquired image data in the storage unit 320 as the image information 322 .

The skeleton recognition unit 332 uses the trained model 321 to recognize the skeleton of the person whose walking posture is to be measured in the image data. For example, the skeleton recognition unit 332 recognizes the skeleton of each piece of time-series image data. The skeleton recognition unit 332 may be configured to recognize the skeleton of image data extracted based on a predetermined condition from time-series image data. For example, the skeleton recognition unit 332 recognizes each part such as the upper part of the spine, the right shoulder, the left shoulder, the right elbow, the left elbow, the right wrist, the left wrist, the right hand, the left hand, and so on. The skeleton recognition unit 332 also calculates the coordinates of the recognized parts in the screen data. Then, the skeleton recognition unit 332 associates the recognition/calculation results with identification information for identifying the image data and stores them in the storage unit 320 as the skeleton information 323 . In this way, the skeleton recognition unit 332 acquires information indicating the position of each part of the person in the image data.

The parts recognized by the skeleton recognition unit 332 correspond to the trained model 321 (teaching data used when learning the trained model 321). Therefore, the skeleton recognition unit 332 may recognize parts other than those exemplified above according to the trained model 321 .

The grounding determination unit 333 determines which foot is in contact with the ground based on the skeleton of the person indicated by the skeleton information 323 . For example, the contact determination unit 333 determines which of the right and left legs of the walking person is on the ground, based on the degree of change in the coordinates of the right and left ankles included in the skeleton information 323. do. For example, the contact determination unit 333 refers to the skeleton information 323 and acquires information indicating the position such as the coordinates of the ankle. Then, when the degree of change in the coordinates of the ankle satisfies a predetermined condition, the ground contact determination unit 333 determines that the foot having the ankle that satisfies the condition is in contact with the ground. Further, the contact determination unit 333 stores information indicating the result of the decision in the storage unit 320 as contact determination information 324 .

FIG. 6 shows an example of changes in the Y coordinates (coordinates in the vertical direction of the image data) of the right and left ankles. For example, in FIG. 6, the X axis indicates values corresponding to time, and the Y axis indicates vertical coordinates (Y coordinates) in image data. For example, in the case of FIG. 6, the larger the Y-axis value, the lower the right or left ankle in the image data (that is, the closer the right or left ankle is to the front of the screen). It is shown that.

In general, when a person walks, one of the right and left feet remains on the ground and the other moves forward, both the right and left feet touch the ground, and one of the right and left feet remains on the ground and one moves forward. , and both right and left feet touch the ground, repeating this exercise alternately. That is, in the case of this embodiment, the Y coordinate of the right ankle is constant and the Y coordinate of the left ankle is increased, the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant, the Y coordinate of the left ankle is constant and the Y coordinate of the right ankle is constant. It is assumed that the state in which the coordinates increase and the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant is repeated.

Therefore, the ground contact determination unit 333 determines which foot is grounded based on the degree of change in the Y coordinates of the right ankle and the left ankle. In other words, the ground contact determination unit 333 determines that the foot having the ankle is in contact with the ground when it can be evaluated that the Y coordinate of the ankle is constant. Specifically, for example, the contact determination unit 333 compares the Y coordinate of the ankle in the image data to be determined with the Y coordinate of the ankle in the image data preceding the determination target in the time-series image data. . Then, when the displacement (amount of change) of the ankle from the Y coordinate in the previous image data is equal to or less than a predetermined value, the contact determination unit 333 assumes that the Y coordinate is constant and determines the ankle with the constant Y coordinate. It is determined that the foot that has the ground is in contact with the ground. Note that the contact determination unit 333 may determine that the Y coordinate is constant by a method other than the above-described example. Further, the range of displacement in which the ground contact determination unit 333 determines that the Y coordinate is constant may be set arbitrarily.

Further, the contact determination unit 333 is configured to determine to exclude from contact determination when it is determined that the leg other than the leg determined to be in contact with the ground has started to move based on the Y-coordinate of the ankle. You may For example, in general walking, the left foot begins to move after the right foot touches the ground. After the Y coordinate of the right ankle becomes constant, the contact determination section 333 may make a decision to exclude from contact determination at the timing when the Y coordinate of the left ankle starts to increase. Specifically, for example, after the Y coordinate of the right ankle becomes constant, the contact determination unit 333 determines that the displacement of the Y coordinate of the left ankle from the previous image data in the time-series image data When the displacement of the Y coordinate of the right ankle from the image data is exceeded, it can be determined that the left foot has started to move and can be excluded from the ground contact determination.

Also, in general, when a person walks, the foot that newly touches the ground advances further than the foot that has already touched the ground. That is, in the case of this embodiment, the Y coordinate of the ankle of the newly grounded foot is larger than the Y coordinate of the ankle of the already grounded foot. Therefore, the contact determination unit 333 may be configured to determine whether or not the foot is contacting the ground by focusing on only the ankle having the larger Y coordinate between the right ankle and the left ankle.

Further, the contact determination unit 333 may be configured to refer to the coordinates of the knee in addition to the coordinates of the ankle when determining the contact. For example, the contact determination unit 333 may be configured to make a decision to exclude from the contact decision when the coordinate change of the knee satisfies a predetermined condition. Specifically, for example, the ground contact determination unit 333 determines whether the displacement of the Y coordinate of the knee from the previous image data in the time-series image data is 0 or less for the leg to be determined whether or not it is grounded. , it may be configured to make a determination to exclude it from the ground contact determination.

For example, as described above, the contact determination unit 333 makes contact determination based on the degree of change in the Y coordinate of the ankle. Note that the contact determination unit 333 may be configured to perform contact determination based on the degree of change in the skeleton other than the above example, such as using the Y coordinate of the toe portion.

The ground contact timing determination unit 334 determines the timing at which the foot touches the ground based on the placement of the foot determined based on the skeleton of the person indicated by the skeleton information 323 . For example, the contact timing determination unit 334 determines the timing at which the foot contacts the ground based on the change in the contacting foot indicated by the contact determination information 324 . Then, the contact timing determination section 334 stores the decision result as the contact timing information 325 in the storage section 320 .

For example, the ground contact timing determination unit 334 determines that the foot has touched the ground at the timing when the grounded foot indicated by the ground contact determination result is switched to another foot. For example, referring to FIG. 7, the "left foot" is grounded in the identification information "124" and "125", and is excluded from the grounding determination in the identification information "126" and "127". After that, it is determined that the "right foot" is on the ground with the identification information "128". That is, in the case of FIG. 7, the foot on the ground is switched from the left foot to the right foot at the timing of the identification information "128". Therefore, the ground contact timing determination unit 334 determines that the right foot has grounded at the timing of the identification information "128".

The output unit 335 outputs contact timing information 325, contact determination information 324, and the like. For example, the output unit 335 can output at least one of the above information to an external device, the smart phone 200, or the like.

The above is a configuration example of the determination device 300 . Next, the operation of the determination device 300 will be described with reference to FIGS. 8 and 9. FIG.

First, with reference to FIG. 8, an operation example of the contact determination unit 333 will be described. Referring to FIG. 8, the contact determination unit 333 compares the Y coordinate of the knee in the image data to be determined with the Y coordinate of the knee in the image data preceding the determination target in the time-series image data ( step S101). If the displacement of the knee from the Y coordinate in the previous image data exceeds 0 (step S101, Yes), the contact determination unit 333 confirms whether or not the displacement of the ankle satisfies the condition (step S102). .

When the displacement of the ankle satisfies the condition (step S102, Yes), the ground contact determination unit 333 determines that the foot satisfying the condition is in contact with the ground (step S103). On the other hand, if the displacement of the Y coordinate of the knee is 0 or less (step S101, No) or if the displacement of the ankle does not satisfy the condition (step S102, No), the ground contact determination unit 333 determines that the foot that does not satisfy the condition is grounded. decide not to.

The above is an operation example of the contact determination unit 333 . For example, when the Y-coordinate of the ankle can be evaluated to be constant based on the degree of change in the Y-coordinate of the ankle, the contact determination unit 333 determines that the leg other than the leg determined to be on the ground starts to move. If no determination is made or if the coordinate change of the knee satisfies the condition, it can be determined that the displacement of the ankle satisfies the condition. Further, the contact determination unit 333 may be configured to identify the ankle of interest based on the Y-coordinate of the ankle and perform the above-described confirmation on the identified ankle.

Next, with reference to FIG. 9, an operation example of the ground contact timing determination section 334 will be described. Referring to FIG. 9, the contact timing determination unit 334 refers to the contact timing information 325 and checks whether or not the contacting foot indicated by the contact decision result has changed (step S201).

When the foot has changed (step S201, Yes), the contact timing determination unit 334 determines that the changed timing is the contact timing (step S202). On the other hand, if the grounded foot indicated by the determination result has not changed (step S201), the ground contact timing determining section 334 does not perform the above determination.

The above is a processing example of the contact timing determination unit 334 .

As described above, the determination device 300 has the skeleton recognition section 332 , the contact determination section 333 and the contact timing determination section 334 . According to such a configuration, the contact timing determination unit 334 determines the timing at which the foot touches the ground based on the determination result of the contacting foot made by the contact determination unit 333 using the coordinates of the skeleton recognized by the skeleton recognition unit 332 . can judge. That is, according to the above configuration, it is possible to grasp the timing when the foot touches the ground based on the image data.

Note that the determination device 300 may be configured to determine the contact timing based on the result of skeleton recognition by an external device that recognizes the skeleton. When configured in this manner, determination device 300 does not need to have the function of skeleton recognition section 332 .

[Second embodiment]
Next, a second embodiment of the present disclosure will be described with reference to FIGS. 10 to 13. FIG. FIG. 10 is a diagram showing a configuration example of the calculation system 400. As shown in FIG. FIG. 11 is a block diagram showing a configuration example of the calculation device 500. As shown in FIG. FIG. 12 is a diagram showing an example of hand length and height. FIG. 13 is a flow chart showing an operation example of the computing device 500 .

In the second embodiment of the present disclosure, a calculation system 400 that calculates an index value indicating the bending of a person's waist in image data based on image data acquired using an imaging device such as the smartphone 200 will be described. . As in the first embodiment, the calculation system 400 acquires image data showing a person walking from the back side to the front side of the screen, and recognizes the position of the skeleton from the image data. Then, the calculation system 400 calculates an index value indicating bending of the waist based on the recognition result.

As will be described later, in the calculation system 400 described in this embodiment, the hand length, which is the sum of the intra-image distances of the skeletal lengths that connect the parts existing between the left and right hands, and the intra-image distance from the head to the toes. Get the height, which is the distance. Then, the calculation system 400 calculates an index value indicating bending of the waist based on the acquired hand length and height. Assuming that both the hand length and the height are the height, the height changes as the waist bends, but the hand length does not change even when the waist is bent. Also, in general, the length of the person when both hands are spread out to the left and right corresponds to the height. The calculation system 400 calculates an index value indicating the bending of the waist by using the above. It is known that the ratio of hand length to height varies depending on age, height, and the like. Therefore, the calculation system 400 adds a predetermined correction value to the hand length in accordance with the attributes of the person such as age, height, gender, etc., and then corrects the bending of the waist based on the corrected hand length and height. It may be configured to calculate an index value to indicate.

Note that the calculation system 400 described in this embodiment can have the function of the determination system 100 described in the first embodiment. That is, the calculation system 400 may have a configuration for determining the contact timing.

FIG. 10 shows a configuration example of the calculation system 400. FIG. Referring to FIG. 10 , the calculation system 400 includes, for example, a smart phone 200 and a calculation device 500 . As shown in FIG. 10, the smartphone 200 and the computing device 500 are connected, for example, wirelessly or by wire so that they can communicate with each other.

The configuration of the smartphone 200 is the same as in the first embodiment. Therefore, description of smartphone 200 is omitted.

The calculation device 500 is an information processing device that calculates an index value indicating bending of the waist based on image data acquired by the smartphone 200 . For example, the computing device 500 is a server device or the like. The computing device 500 may be a single information processing device, or may be implemented on a cloud, for example.

FIG. 11 shows a configuration example of the calculation device 500. FIG. Referring to FIG. 11, the calculation device 500 has, for example, a communication I/F section 510, a storage section 520, and an arithmetic processing section 530 as main components.

The communication I/F unit 510 consists of a data communication circuit. Communication I/F unit 510 performs data communication with an external device, smartphone 200, or the like connected via a communication line.

The storage unit 520 is a storage device such as a hard disk or memory. The storage unit 520 stores processing information and programs 527 necessary for various processes in the arithmetic processing unit 530 . The program 527 realizes various processing units by being read into the arithmetic processing unit 530 and executed. The program 527 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 510 and stored in the storage unit 520 . Main information stored in the storage unit 520 includes, for example, a trained model 521, image information 522, skeleton information 523, hand length information 524, height information 525, waist bending value information 526, and the like.

The learned model 521, image information 522, and skeleton information 523 are the same information as the learned model 321, image information 322, and skeleton information 323 described in the first embodiment.

The hand length information 524 indicates the hand length, which is the sum of the in-image distances of the skeletal lengths connecting the parts existing between the left and right hands, such as palms, wrists, elbows, and shoulders. The hand length information 524 is calculated in advance, for example, based on image data acquired with the hand extended downward so that the hand length is the longest, and is transmitted via the communication I/F unit 510 or the like. It is acquired from an external device or the like and stored in the storage unit 520 . The effort length information 524 may be generated/updated as a result of calculation processing by the effort length calculator 533 .

For example, the effort length information 524 includes information indicating the effort length. The hand length information 524 may include information indicating the hand length acquired for each time-series image data. For example, the effort length information 524 may be information in which identification information indicating image data used when calculating the effort length and information indicating the effort length are associated with each other.

The height information 525 indicates the height of the person in the image data, which is the intra-image distance connecting the head (forehead) and the toes (for example, the lower one of the left and right). The height information 525 is generated/updated as a result of calculation processing by the height calculator 534, for example.

For example, the height information 525 includes information indicating the height acquired for each time-series image data. The height information 525 may be, for example, a correspondence between identification information indicating image data used when calculating the height and information indicating the height. Also, the height information 525 may be information that associates identification information indicating image data that satisfies a predetermined condition with information indicating height.

The waist curvature value information 526 indicates the waist curvature value, which is an index value indicating the curvature of the waist. The waist curvature value information 526 is generated and updated as a result of calculation processing by the curvature calculator 535, for example.

For example, the waist curvature value information 526 includes information indicating the waist curvature value. Also, the waist curvature value information 526 may include information indicating the waist curvature value acquired for each time-series image data. The waist curvature value information 526 may be, for example, a correspondence between identification information indicating image data used in calculating the waist curvature value and information indicating the waist curvature value. Also, the waist curvature value information 526 may be information that associates identification information indicating image data that satisfies a predetermined condition with information indicating the waist curvature value.

The arithmetic processing unit 530 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 530 reads the program 527 from the storage unit 520 and executes it, so that the hardware and the program 527 cooperate to realize various processing units. Main processing units realized by the arithmetic processing unit 530 include, for example, an image acquisition unit 531, a skeleton recognition unit 532, a hand length calculation unit 533, a height calculation unit 534, a bend calculation unit 535, and an output unit 536 .

The image acquisition unit 531 and skeleton recognition unit 532 can perform the same processing as the image acquisition unit 331 and skeleton recognition unit 332 described in the first embodiment.

Based on the skeleton information 523, the hand length calculation unit 533 calculates the hand length, which is the sum of the in-image distances of the skeleton lengths connecting the left and right hands. Then, the effort length calculation unit 533 stores the calculated effort length as the effort length information 524 in the storage unit 520 .

For example, FIG. 12 shows an example of the hand length. Referring to FIG. 12 , hand length calculation unit 533 obtains information indicating the positions of parts such as palms, wrists, elbows, and shoulders between the left and right hands by referring to skeleton information 523 . Then, based on the acquired information, the hand length calculation unit 533 calculates the total sum of the distances between adjacent parts when connecting the adjacent parts with a straight line. For example, as described above, the hand length calculator 533 calculates the hand length based on the position of each part between the left and right hands.

The effort length calculation unit 533 can perform the above-described effort length calculation processing for each piece of time-series image data. In other words, the effort length calculation unit 533 may be configured to calculate the effort length for each image data. The effort length calculator 533 may be configured to calculate the effort length in image data extracted under arbitrary conditions from time-series image data.

It should be noted that the effort length calculation unit 533 may calculate the effort length using parts other than those exemplified above. Also, the information indicating the length of effort can be calculated and acquired in advance. If the information indicating the length of effort is acquired in advance, the calculation device 500 does not need to have the length-of-handling calculation section 533 .

The height calculation unit 534 calculates the height, which is the in-image distance from the head to the toes, based on the skeleton information 523 . Height calculator 534 then stores the calculated height as height information 525 in storage 520 .

For example, FIG. 12 shows an example of height. Referring to FIG. 12, the height calculation unit 534 obtains information indicating the positions of the forehead and the left and right toes, which are the highest parts, by referring to the skeleton information 523 . Then, the height calculation unit 534 calculates the distance obtained by connecting the forehead and the lower one of the left and right toes with a straight line based on the acquired information. Alternatively, the height calculation unit 534 calculates the difference in the Y-axis direction (vertical direction in FIG. 12) between the forehead and the lower one of the left and right toes based on the acquired information. For example, as described above, the height calculator 534 calculates the height based on the positions of the head and feet.

The height calculation unit 534 can perform the above-described height calculation processing on each piece of time-series image data, similar to the effort length calculation unit 533 . In other words, the height calculator 534 may be configured to calculate the height of each piece of image data. The height calculator 534 may be configured to calculate the height of image data extracted under arbitrary conditions from time-series image data.

Note that the height calculation unit 534 may calculate the height using parts other than those exemplified above. For example, the height calculation unit 534 may calculate the length between parts including the waist between the neck and the ankles as the height.

The curvature calculation unit 535 calculates a waist curvature value, which is an index value indicating the curvature of the waist, based on the length of hand indicated by the length of hand information 524 and the height indicated by the height information 525 . Then, the curvature calculation unit 535 stores the calculated waist curvature value in the storage unit 520 as the waist curvature value information 526 .

For example, the curvature calculation unit 535 calculates the waist curvature value by dividing the hand length by the height. That is, the curvature calculation unit 535 calculates the waist curvature value by calculating (hand length)/(height). For example, the curvature calculation unit 535 can calculate the waist curvature value using the hand length and height calculated for each image data.

In addition, it is assumed that the length of effort will be shortened by waving your hands while walking. Therefore, the bend calculation unit 535 uses the length of hand selected from the length of hand information 524 based on a predetermined standard, such as using the longest length of hand among the lengths of hand included in the length of hand information 524. A curvature value may be calculated. In other words, the hand length used by the bending calculator 535 to calculate the waist bending value may be a hand length selected according to a predetermined standard from among the hand lengths included in the hand length information 524. . The criteria for selection may be set arbitrarily, such as using the longest one. Also, the effort length information 524 may include only one previously acquired effort length. Therefore, the hand length used when the curvature calculator 535 calculates the waist curvature value may be, for example, a constant value.

Further, the curvature calculation unit 535 may use the determination result of the contact timing described in the first embodiment to specify image data for which the waist curvature value is to be calculated. For example, the curvature calculation unit 535 may be configured to calculate the waist curvature value using the height acquired based on the image data determined to be the contact timing. The curvature calculation unit 535 may be configured to calculate the curvature value of the waist using the height acquired based on the image data for a predetermined number of frames from the image data determined to be the contact timing. In this way, by configuring to calculate the bending value of the waist (based on the height) based on the image data specified based on the determination result of the contact timing, the index to be compared with other people can be obtained. , a more appropriate waist bending value can be calculated. The curvature calculation unit 535 may be configured to calculate a waist curvature value for each piece of time-series image data and to calculate an average value of the calculated waist curvature values.

The output unit 536 outputs hand length information 524, height information 525, waist bending value information 526, and the like. For example, the output unit 536 can output at least one of the above information to an external device, the smart phone 200, or the like.

The above is an example of the configuration of the calculation device 500. Next, an operation example of the calculation device 500 will be described with reference to FIG. 13 .

Referring to FIG. 13, the bend calculation unit 535 acquires the length of effort indicated by the length of effort information 524 and the height indicated by the height information 525 (step S301).

The curvature calculation unit 535 calculates a waist curvature value, which is an index value indicating the curvature of the waist, based on the hand length and height (step S302). For example, the curvature calculation unit 535 calculates the waist curvature value by dividing the hand length by the height.

The above is an operation example of the calculation device 500 .

Thus, the calculation device 500 has the bend calculation section 535 . According to such a configuration, the calculation device 500 can calculate the waist bending value based on the acquired hand length and height. That is, according to the above configuration, it is possible to calculate the index value corresponding to the bending of the waist based on the image data.

Note that the calculation device 500 can have the function of the determination device 300 described in the first embodiment. Further, the calculation device 500 can have modifications similar to the determination device 300 described in the first embodiment.

[Third embodiment]
Next, a third embodiment of the present disclosure will be described with reference to FIGS. 14 to 19. FIG. FIG. 14 is a diagram showing a configuration example of the calculation system 600. As shown in FIG. FIG. 15 is a block diagram showing a configuration example of the calculation device 700. As shown in FIG. FIG. 16 is a diagram showing an example of height. 17 and 18 are diagrams showing examples of leg components. FIG. 19 is a flow chart showing an operation example of the computing device 700 .

In the third embodiment of the present disclosure, a calculation system 600 that calculates an index value indicating the state of a target of interest based on image data acquired using an imaging device such as the smartphone 200 will be described. As in the first and second embodiments, the calculation system 600 acquires image data showing how a person walks from the back side to the front side of the screen, and calculates the position of the skeleton from the image data. recognize. Calculation system 600 then calculates an index value indicating the state of the object based on the recognition result.

For example, in the calculation system 600 described in the present embodiment, focusing on the foot, an index value indicating the state of the toe, such as the toe's rise and direction, is calculated. Specifically, for example, the calculation system 600 calculates the difference in the X-axis direction and the difference in the Y-axis direction between the positions of the ankle and the tip of the foot, which are parts present in the foot of interest, and the comparative length to be compared. and get. Then, the calculation system 600 calculates an index value indicating the direction of the toe based on the difference in the X-axis direction and the comparative length, and an index value indicating the rise of the toe based on the difference in the Y-axis direction and the comparative length. Calculate People and the like appear larger as they get closer to the camera, which is an imaging device. Therefore, even if the toes are similarly raised or pointed in the same direction, the length in the image data differs according to the proximity of the person to the imaging device. The calculation system 600 calculates an index value that solves the above problem by comparing the difference in the X-axis direction and the difference in the Y-axis direction between the positions of the ankle and the toe with the comparison length, which is the length to be compared. make it possible.

Note that, in the present embodiment, a case where attention is focused on the foot as the target of attention will be exemplified. However, the target of interest may be any location other than the foot. In other words, the calculation system 600 may use the difference in the X-axis direction and the difference in the Y-axis direction of the positions of parts existing in the object of interest other than the ankles and toes. In addition, the comparative length is the length between parts where the length in the image data varies depending on the proximity of the person to the camera, which is the imaging device, when the image data is acquired, but the length is constant in the person. It means that. The comparative lengths are, for example, the height, which is the distance in the image from the head to the toes described in the second embodiment (the bending of the waist is not considered), the hand length, and the image from the head to the pelvis. One of height, which is the inner distance, and so on can be used. The comparative length may be a length between sites other than those exemplified above.

Also, the calculation system 600 described in this embodiment can have the functions of the determination system 100 described in the first embodiment and the calculation system 400 described in the second embodiment. In other words, the calculation system 600 may have a configuration for determining contact timing, a configuration for calculating a waist bending value, and the like.

FIG. 14 shows a configuration example of the calculation system 600. FIG. Referring to FIG. 14, the calculation system 600 has, for example, a smart phone 200 and a calculation device 700 . As shown in FIG. 14, the smartphone 200 and the computing device 700 are connected, for example, wirelessly or by wire so that they can communicate with each other.

The configuration of the smartphone 200 is the same as in the first embodiment. Therefore, description of smartphone 200 is omitted.

The calculation device 700 is an information processing device that calculates, based on the image data acquired by the smartphone 200, an index value indicating the toe lift and the toe direction. For example, the computing device 700 is a server device or the like. The computing device 700 may be a single information processing device, or may be implemented on a cloud, for example. Calculation device 700 may calculate only one of the index value indicating the toe-up and the index value indicating the direction of the toe.

FIG. 15 shows a configuration example of the calculation device 700. FIG. Referring to FIG. 15, the calculation device 700 has, for example, a communication I/F section 710, a storage section 720, and an arithmetic processing section 730 as main components.

The communication I/F unit 710 consists of a data communication circuit. Communication I/F unit 710 performs data communication with an external device, smartphone 200, or the like connected via a communication line.

The storage unit 720 is a storage device such as a hard disk or memory. The storage unit 720 stores processing information and programs 727 necessary for various processes in the arithmetic processing unit 730 . The program 727 realizes various processing units by being read into the arithmetic processing unit 730 and executed. The program 727 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 710 and stored in the storage unit 720 . Main information stored in the storage unit 720 includes, for example, a trained model 721, image information 722, skeleton information 723, comparative length information 724, foot component information 725, toe data information 726, and the like.

The learned model 721, image information 722, and skeleton information 723 are the same information as the learned model 321, image information 322, and skeleton information 323 described in the first embodiment.

The comparative length information 724 is the length between parts that is constant in the person, although the length in the image data varies depending on the proximity of the person to the camera, which is the imaging device, when the image data is acquired. It shows the comparative length, which is the length. For example, the comparative length information 724 indicates one of the height, which is the distance in the image from the head to the toes, the hand length, the second height, which is the distance in the image from the head to the pelvis, and so on. ing. The comparative length information 724 is generated and updated as a result of calculation processing by the comparative length calculator 733, for example.

For example, the comparison length information 724 includes information indicating the comparison length acquired for each time-series image data. For example, the comparison length information 724 may be information that associates identification information indicating image data used when calculating the comparison length with information indicating the comparison length.

The foot component information 725 indicates inter-part lengths such as the X component, which is the difference between the positions of the ankle and the tip of the foot in the X-axis direction, and the Y component, which is the difference between the positions of the ankle and the tip of the foot in the Y-axis direction. The foot component information 725 is generated and updated as a result of calculation processing by the foot component calculator 734, for example. Note that the foot component information 725 may include only one of the X component and the Y component.

For example, the foot component information 725 includes information indicating the X component and the Y component acquired for each time-series image data. For example, the comparison length information 724 may be information that associates identification information indicating image data used when calculating the X component and the Y component with information indicating the X component and the Y component.

The toe data information 726 indicates an index value indicating the rise of the toe and an index value indicating the direction of the toe. The toe data information 726 is generated and updated as a result of calculation processing by the toe data calculator 735, for example.

For example, the toe data information 726 includes an index value indicating the toe lift and an index value indicating the direction of the toe. The toe data information 726 includes an index value indicating the rise of the toe and an index value indicating the direction of the toe acquired for each time-series image data. The height information 525 is, for example, information in which identification information indicating image data used when calculating an index value is associated with information indicating an index value indicating a raised toe or an index value indicating the direction of the toe. It's okay. Also, the toe data information 726 is information that associates identification information indicating image data that satisfies a predetermined condition with information indicating an index value indicating a raised toe or an index value indicating the direction of the toe. good too.

The arithmetic processing unit 730 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 730 reads the program 727 from the storage unit 720 and executes it, thereby realizing various processing units by cooperating the hardware and the program 727 . Main processing units realized by the arithmetic processing unit 730 include, for example, an image acquisition unit 731, a skeleton recognition unit 732, a comparison length calculation unit 733, a foot component calculation unit 734, and a toe data calculation unit 735. , output 736, and so on.

The image acquisition unit 731 and skeleton recognition unit 732 can perform the same processing as the image acquisition unit 331 and skeleton recognition unit 332 described in the first embodiment.

Based on the skeleton information 723, the comparative length calculation unit 733 calculates the height, which is the distance in the image from the head to the toes, the hand length, the second height, which is the distance in the image from the head to the pelvis, and the like. A comparison length, which is one of them, is calculated. Then, the comparison length calculator 733 stores the calculated comparison length as the comparison length information 724 in the storage unit 720 .

For example, FIG. 16 shows an example of height, which is an example of comparative length. Referring to FIG. 16, the comparative length calculation unit 733 obtains information indicating the positions of the forehead and the left and right toes, which are the highest parts, by referring to the skeleton information 523 . Then, the comparative length calculator 733 calculates the distance obtained by connecting the forehead and the lower one of the left and right toes with a straight line based on the acquired information. Alternatively, the comparative length calculator 733 calculates the difference in the Y-axis direction (vertical direction in FIG. 12) between the forehead and the lower one of the left and right toes based on the acquired information. For example, as described above, the comparative length calculator 733 calculates the height based on the positions of the head and feet. Note that the comparison length calculation unit 733 may be configured to calculate, as a comparison length, a hand length, a second height that is an in-image distance from the head to the pelvis, and the like, through similar processing.

The foot component calculation unit 734 refers to the skeleton information 523 to calculate the X component, which is the difference between the positions of the ankle and the tip of the foot in the X-axis direction, and the Y component, which is the difference between the positions of the ankle and the tip of the foot in the Y-axis direction. at least one of the site-to-site lengths of Then, foot component calculation section 734 stores the calculated foot component as foot component information 725 in storage section 720 .

For example, FIG. 17 shows an example of the Y component. Referring to FIG. 17 , the foot component calculation unit 734 obtains information indicating the positions of the ankle and the toe by referring to the skeleton information 523 . Then, the foot component calculator 734 calculates the difference between the ankle and the toe in the Y-axis direction (vertical direction in FIG. 17) based on the acquired information. For example, as described above, the foot component calculator 734 calculates the Y component based on the positions of the ankle and the tip of the foot.

Also, FIG. 18 shows an example of the X component. Referring to FIG. 18, foot component calculation section 734 obtains information indicating the positions of the ankle and toe by referring to skeleton information 523 . Then, the foot component calculator 734 calculates the difference between the ankle and the toe in the X-axis direction (horizontal direction in FIG. 18) based on the acquired information. For example, as described above, the foot component calculator 734 calculates the X component based on the ankle and toe positions.

Note that the foot component calculation unit 734 may be configured, for example, to calculate the Y component and the X component of each of the left and right feet, or only the Y component and the X component of the left and right foot that are positioned below. It may be configured to calculate the components. Further, the foot component calculation unit 734 may be configured to determine whether or not to calculate the foot component using the determination result of the contact timing described in the first embodiment. For example, the foot component calculator 734 can be configured to calculate the Y component and the X component based on the image data determined as the contact timing. The foot component calculator 734 may be configured to calculate the Y component and the X component based on a predetermined frame of image data from the image data determined to be the contact timing.

The toe data calculation unit 735 calculates an index value indicating the rise of the toe based on the comparison length indicated by the comparison length 724 and the Y component indicated by the foot component information 725. Also, the toe data calculator 735 calculates an index value indicating the direction of the toe based on the comparative length indicated by the comparative length 724 and the X component indicated by the foot component information 725 . Then, the toe data calculation unit 735 stores the calculated index value indicating the upward movement of the toe and the calculated index value indicating the direction of the toe in the storage unit 720 as the toe data information 726 .

For example, the toe data calculation unit 735 calculates an index value indicating the rise of the toe by dividing the Y component by the comparison length. That is, the toe data calculation unit 735 calculates the index value indicating the rise of the toe by calculating (Y component)/(comparative length). For example, the toe data calculation unit 735 can calculate an index value indicating the rise of the toe using the Y component and the comparison length calculated for each image data.

Also, for example, the toe data calculation unit 735 calculates an index value indicating the direction of the toe by dividing the X component by the comparison length. That is, the toe data calculator 735 calculates an index value indicating the direction of the toe by calculating (X component)/(comparative length). For example, the toe data calculation unit 735 can calculate an index value indicating the direction of the toe using the X component and the comparison length calculated for each image data.

Note that the toe data calculation unit 735 uses the determination result of the contact timing described in the first embodiment to calculate the image data for which the index value indicating the toe lift and the index value indicating the direction of the toe are to be calculated. may be specified. For example, the toe data calculation unit 735 calculates an index value indicating the rise of the toe and an index value indicating the direction of the toe using the Y component and the X component obtained based on the image data determined as the contact timing and the comparison length. can be configured to The toe data calculation unit 735 calculates an index value indicating the rise of the toe and the direction of the toe by using the Y component and the X component obtained based on the image data for a predetermined frame from the image data determined as the contact timing and the comparison length. It may be configured to calculate an index value to indicate. In this way, by performing calculation based on image data specified based on the determination result of the contact timing, it becomes possible to evaluate the toe lift and the direction of the toe when the foot touches the ground.

The output unit 736 outputs comparative length information 724, foot component information 725, toe data information 726, and the like. For example, the output unit 736 can output at least one of the above information to an external device, the smartphone 200, or the like.

The above is a configuration example of the calculation device 700 . Next, an operation example of the calculation device 700 will be described with reference to FIG. 19 .

Referring to FIG. 19, the toe data calculation unit 735 acquires the comparative length indicated by the comparative length information and the Y component and the X component indicated by the foot component information 725 (step S401).

The toe data calculation unit 735 calculates toe data based on the comparison length and the Y component and the X component. For example, the toe data calculation unit 735 calculates an index value indicating the rise of the toe based on the comparison length and the Y component. Also, for example, the toe data calculation unit 735 calculates an index value indicating the direction of the toe based on the comparison length and the X component.

The above is an operation example of the calculation device 700 .

Thus, the computing device 700 has a toe data computing section 735. According to such a configuration, the calculation device 700 can calculate the index value indicating the toe-up and the index value indicating the direction of the toe based on the obtained comparison length and the Y component and the X component. In other words, according to the above configuration, it is possible to calculate an index value indicating a state such as the toe rising or the direction based on the image data.

Note that when the target of interest is an arbitrary part other than the foot, the calculation device 700 is configured to calculate the length between parts existing in the target of interest other than the ankle and the toe as the Y component and the X component. you can

Also, the calculation device 700 can have a function as the determination device 300 described in the first embodiment. Further, the calculation device 700 can have modifications similar to the determination device 300 described in the first embodiment. Similarly, the computing device 700 can have the function of the computing device 500 described in the second embodiment. Also, the computing device 700 can have modifications similar to the computing device 500 described in the second embodiment.

[Fourth embodiment]
Next, a fourth embodiment of the present disclosure will be described with reference to FIGS. 20 to 22. FIG. In the fourth embodiment of the present disclosure, an overview of the configuration of a computing device 800, which is an information processing device, will be described.

FIG. 20 shows a hardware configuration example of the computing device 800 . Referring to FIG. 20, the computing device 800 has, as an example, the following hardware configuration.
- CPU (Central Processing Unit) 801 (arithmetic unit)
・ROM (Read Only Memory) 802 (storage device)
・RAM (Random Access Memory) 803 (storage device)
Program group 804 loaded into RAM 803
- Storage device 805 for storing program group 804
- A drive device 806 that reads and writes a recording medium 810 external to the information processing device
- A communication interface 807 that connects to a communication network 811 outside the information processing apparatus
An input/output interface 808 for inputting/outputting data
A bus 809 connecting each component

In addition, the calculation device 800 can realize the functions of the height calculation unit 821 and the curve calculation unit 822 shown in FIG. The program group 804 is stored in the storage device 805 or the ROM 802 in advance, for example, and is loaded into the RAM 803 or the like and executed by the CPU 801 as necessary. The program group 804 may be supplied to the CPU 8801 via the communication network 811 or may be stored in the recording medium 810 in advance, and the drive device 806 may read the program and supply it to the CPU 801 .

Note that FIG. 20 shows a hardware configuration example of the computing device 800 . The hardware configuration of the computing device 800 is not limited to the case described above. For example, the computing device 800 may consist of some of the configurations described above, such as not having the drive device 806 .

The height calculator 821 calculates the height of the person in the image data based on the image data.

The bend calculation unit 822 acquires the hand length, which is the length connecting the left and right hands of the person in the image data. Also, the curve calculation unit 822 calculates an index value indicating the curve of the waist of the person in the image data based on the obtained hand length and the height calculated by the height calculation unit 821 .

Thus, the calculation device 800 has a height calculation section 821 and a curvature calculation section 822. In addition, the bend calculator 822 is configured to acquire the hand length. According to such a configuration, the curve calculation unit 822 calculates an index value indicating the curve of the waist of the person in the image data based on the obtained hand length and the height calculated by the height calculation unit 821. can do As a result, it is possible to calculate an index value corresponding to the bending of the waist based on the image data.

An information processing device such as the computing device 800 described above can be realized by installing a predetermined program in the information processing device. Specifically, a program, which is another aspect of the present invention, causes an information processing apparatus to calculate the height of a person in the image data based on the image data, and connect the left and right hands of the person in the image data. A program for realizing a process of acquiring a hand length, which is a length, and calculating an index value indicating the bending of the waist of a person in image data based on the acquired hand length and the calculated height. is.

Further, in the calculation method executed by the information processing device described above, the information processing device calculates the height of the person in the image data based on the image data, and connects the right and left hands of the person in the image data. The hand length, which is the length, is acquired, and an index value indicating the bending of the waist of the person in the image data is calculated based on the acquired hand length and the calculated height.

Even in the invention of the program (or recording medium) or calculation method having the configuration described above, the calculation device 800 having the curvature calculation section 822 described above has the same functions and effects as those described above. can achieve the same purpose.

Calculation device 800 realizes the functions of component calculation unit 831, comparison length calculation unit 832, and index value calculation unit 833 shown in FIG. may be configured to

The component calculation unit 831 calculates, based on the image data, the inter-part length between the parts present in the target of interest in the person in the image data.

The comparison length calculator 832 calculates a comparison length to be compared based on the image data.

The index value calculation unit 833 calculates an index value indicating the state of the object of interest based on the inter-part length calculated by the component calculation unit 831 and the comparison length calculated by the comparison length calculation unit 832 .

In this way, the calculation device 800 can have a component calculation section 831 , a comparison length calculation section 832 and an index value calculation section 833 . According to such a configuration, the index value calculation unit 833 calculates the index value indicating the state of the object of interest based on the inter-part length calculated by the component calculation unit 831 and the comparison length calculated by the comparison length calculation unit 832. value can be calculated. As a result, it becomes possible to calculate an index value corresponding to the state of the target of interest based on the image data.

An information processing device such as the computing device 800 described above can be realized by installing a predetermined program in the information processing device. Specifically, a program, which is another aspect of the present invention, causes an information processing device to calculate, based on image data, an inter-part length between parts existing in a target of interest in a person in the image data, Based on the image data, a comparative length to be compared is calculated, and based on the calculated length between parts and the calculated comparative length, an index value indicating the state of the target of interest is calculated. It is a program for

Further, in the calculation method executed by the information processing device described above, the information processing device calculates, based on the image data, the inter-part length between the parts existing in the target of interest in the person in the image data, Based on the image data, a comparative length to be compared is calculated, and an index value indicating the state of the object of interest is calculated based on the calculated length between parts and the calculated comparative length. .

Even in the invention of the program (or recording medium) or calculation method having the above-described configuration, the calculation device having the above-described index value calculation unit 833 in order to have the same action and effect as the above-described case. It can achieve the same purpose as the 800.

<Appendix>
Some or all of the above embodiments may also be described as the following appendices. An outline of the calculation device and the like in the present invention will be described below. However, the present invention is not limited to the following configurations.

(Appendix 1)
a height calculation unit that calculates the height of a person in the image data based on the image data;
A hand length, which is a length connecting the left and right hands of a person in the image data, is obtained, and based on the obtained hand length and the height calculated by the height calculation unit, a curvature calculation unit that calculates an index value indicating the curvature of the waist of the person;
A computing device.
(Appendix 2)
The calculation device according to appendix 1, wherein the curve calculation unit calculates the index value by dividing the hand length by the height.
(Appendix 3)
a hand length calculation unit that calculates the hand length of the person in the image data based on the image data;
According to appendix 1 or appendix 2, the bend calculation unit calculates the index value based on the hand length calculated by the hand length calculation unit and the height calculated by the height calculation unit. calculator.
(Appendix 4)
The calculation device according to appendix 3, wherein the hand length calculation unit calculates the hand length based on information indicating coordinates of each part of the person in the image data recognized based on the image data.
(Appendix 5)
The bend calculation unit calculates the index based on the length of effort selected according to a predetermined criterion from among the lengths of effort calculated by the length of effort calculation unit and the height calculated by the height calculation unit. The calculation device according to appendix 3 or appendix 4, which calculates a value.
(Appendix 6)
The bend calculation unit acquires the length of effort stored in advance in a storage device, and calculates the index value based on the acquired length of effort and the height calculated by the height calculation unit. The calculation device according to appendix 1 or appendix 2.
(Appendix 7)
The bend calculation unit obtains contact timing information indicating the timing at which the person's foot contacts the ground in the image data, and calculates the height calculated based on the image data specified based on the contact timing information. The calculation device according to any one of appendices 1 to 6, wherein the index value is calculated based on the
(Appendix 8)
The height calculation unit calculates the height based on information indicating coordinates of each part of the person in the image data recognized based on the image data. 2. The computing device according to item 1.
(Appendix 9)
The information processing device
Based on the image data, calculate the height of the person in the image data,
A hand length, which is a length connecting the left and right hands of the person in the image data, is obtained, and the curve of the waist of the person in the image data is calculated based on the obtained hand length and the calculated height. Calculation method for calculating an index value that indicates
(Appendix 10)
information processing equipment,
Based on the image data, calculate the height of the person in the image data,
A hand length, which is a length connecting the left and right hands of the person in the image data, is obtained, and the curve of the waist of the person in the image data is calculated based on the obtained hand length and the calculated height. A program for realizing processing that calculates an index value that indicates

The programs described in each of the above embodiments and supplementary notes are stored in a storage device or recorded in a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Although the present invention has been described with reference to the above-described embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

In addition, the present invention enjoys the benefit of the priority claim based on the patent application of Japanese Patent Application No. 2021-055313 filed on March 29, 2021 in Japan, and is described in the patent application. The contents are hereby incorporated by reference in their entirety.

100 determination system 200 smart phone 300 determination device 310 communication I/F unit 320 storage unit 321 trained model 322 image information 323 skeleton information 324 contact determination information 325 contact timing information 326 program 330 arithmetic processing unit 331 image acquisition unit 332 skeleton recognition unit 333 Landing determination unit 334 Landing timing determination unit 335 Output unit 400 Calculation system 500 Calculation device 510 Communication I/F unit 520 Storage unit 521 Trained model 522 Image information 523 Skeleton information 524 Hand length information 525 Height information 526 Waist bending value information 527 program 530 arithmetic processing unit 531 image acquisition unit 532 skeleton recognition unit 533 hand length calculation unit 534 height calculation unit 535 bend calculation unit 536 output unit 600 calculation system 700 calculation device 710 communication I/F unit 720 storage unit 721 learning Finished model 722 Image information 723 Skeleton information 724 Comparison length information 725 Foot component information 726 Toe data information 727 Program 730 Operation processing unit 731 Image acquisition unit 732 Skeleton recognition unit 733 Comparison length calculation unit 734 Foot component calculation unit 735 Toe data calculation Unit 736 Output unit 800 Calculation device 801 CPU
802 ROMs
803 RAM
804 program group 805 storage device 806 drive device 807 communication interface 808 input/output interface 809 bus 810 recording medium 811 communication network 821 height calculator 822 bend calculator 831 component calculator 832 comparative length calculator 833 index value calculator

Claims (10)

1. a height calculation unit that calculates the height of a person in the image data based on the image data;
   A hand length, which is a length connecting the left and right hands of a person in the image data, is obtained, and based on the obtained hand length and the height calculated by the height calculation unit, a curvature calculation unit that calculates an index value indicating the curvature of the waist of the person;
   A computing device.
2. The calculation device according to claim 1, wherein the curve calculation unit calculates the index value by dividing the hand length by the height.
3. a hand length calculation unit that calculates the hand length of the person in the image data based on the image data;
   3. According to claim 1 or claim 2, the curve calculation unit calculates the index value based on the hand length calculated by the hand length calculation unit and the height calculated by the height calculation unit. The computing device described.
4. 4. The calculation device according to claim 3, wherein the hand length calculator calculates the hand length based on information indicating coordinates of each part of the person in the image data recognized based on the image data.
5. The bend calculation unit calculates the index based on the length of effort selected according to a predetermined criterion from among the lengths of effort calculated by the length of effort calculation unit and the height calculated by the height calculation unit. 5. A calculation device according to claim 3 or 4, for calculating a value.
6. The bend calculation unit acquires the length of effort stored in advance in a storage device, and calculates the index value based on the acquired length of effort and the height calculated by the height calculation unit. 3. A computing device according to claim 1 or claim 2.
7. The bend calculation unit obtains contact timing information indicating the timing at which the person's foot contacts the ground in the image data, and calculates the height calculated based on the image data specified based on the contact timing information. 7. The calculation device according to any one of claims 1 to 6, wherein the index value is calculated based on the
8. The height calculator calculates the height based on information indicating the coordinates of each part of the person in the image data recognized based on the image data. A computing device according to any one of the preceding claims.
9. The information processing device
   Based on the image data, calculate the height of the person in the image data,
   A hand length, which is a length connecting the left and right hands of the person in the image data, is obtained, and the curve of the waist of the person in the image data is calculated based on the obtained hand length and the calculated height. Calculation method for calculating an index value that indicates
10. information processing equipment,
    Based on the image data, calculate the height of the person in the image data,
    A hand length, which is a length connecting the left and right hands of the person in the image data, is obtained, and the curve of the waist of the person in the image data is calculated based on the obtained hand length and the calculated height. A computer-readable recording medium in which a program for realizing processing for calculating an index value indicating is recorded.
    
    

Images