WO2022209286A1

WO2022209286A1 - Determination device

Info

Publication number: WO2022209286A1
Application number: PCT/JP2022/004470
Authority: WO
Inventors: 宏紀寺島; 克幸永井
Original assignee: Ｎｅｃソリューションイノベータ株式会社
Priority date: 2021-03-29
Filing date: 2022-02-04
Publication date: 2022-10-06
Also published as: CN117119958A; JPWO2022209286A1

Abstract

A determination device 400 has a determination unit 421 that acquires information acquired from each piece of time-series image data and indicating the position where a predetermined portion exists, and determines a timing when the foot contacts the ground on the basis of a change in the acquired position at which the portion exists.

Description

judgment device

The present invention relates to a determination device, a determination method, and a recording medium.

It is known to perform user gait analysis based on data.

For example, Patent Document 1 is a document describing human gait analysis. In Patent Document 1, a data acquisition unit that acquires two types of image data from a depth sensor, a skeleton information creation unit that creates skeleton information based on the image data acquired by the data acquisition unit, and a skeleton information creation unit that creates A gait analysis device having a correction processing unit that corrects skeleton information and an analysis processing unit that analyzes a user's gait using the corrected skeleton information is described.

WO2017/170832

When performing various gait analyses, it is necessary to grasp the timing when the foot touches the ground. However, for example, when performing analysis based on image data without using a depth sensor as described in Patent Literature 1, since information in the depth direction cannot be obtained, it is difficult to grasp the timing of ground contact. rice field. Specifically, for example, in Japanese Unexamined Patent Application Publication No. 2002-100000, the contact timing is set to the frame when the displacement of the Y value of the thoracic and lumbar region, which is regarded as a three-dimensional position, changes from a negative value to a positive value. However, when performing analysis based on image data acquired from one direction without using a depth sensor or the like, it is impossible to acquire information in the depth direction, and therefore the skeleton cannot be captured three-dimensionally. As a result, for example, in the case of image data of a person photographed from the side, the arms and shoulders closer to the camera than the thoracic and lumbar region interfere with recognition of the thoracic and lumbar region, making it difficult to recognize the thoracic and lumbar region. In this case, since the size of a person walking from the back of the screen to the front changes on the screen between the back and front, the method using the displacement of the thoracic and lumbar region as described in Patent Document 1 can be used as it is. was gone.

As a method of grasping the touchdown timing, for example, there is also a method of judging by laying a pressure gauge on the floor. However, in the case of such a method, the measurable section is limited to the size of the pressure gauge. Therefore, it is impossible to grasp the contact timing without limiting the measurable section.

Therefore, an object of the present invention is to provide a determination device, a determination method, and a recording medium that solve the problem that it is difficult to grasp the timing at which a foot touches the ground based on image data.

In order to achieve such an object, the determination device, which is one aspect of the present disclosure,
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. It has a configuration of having a determination unit.

In addition, a determination method, which is another form of the present disclosure,
The information processing device
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. take the configuration.

In addition, a recording medium that is another aspect of the present disclosure includes:
information processing equipment,
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. It is a computer-readable recording medium in which a program for realizing processing is recorded.

According to each configuration as described above, it is possible to grasp the timing at which the foot touches the ground based on the image data.

1 is a diagram illustrating a configuration example of a determination system according to a first embodiment of the present disclosure; FIG. FIG. 4 is a diagram for explaining an example of image data acquired by the determination system; FIG. 2 is a block diagram showing a configuration example of a determination device shown in FIG. 1; FIG. FIG. 4 is a diagram showing an example of skeleton information; It is a figure which shows an example of grounding determination information. FIG. 10 is a diagram for explaining an example of determination by a contact determination unit; FIG. 4 is a diagram for explaining an example of determination by a touch timing determination unit; 5 is a flow chart showing an example of processing by a ground contact determination unit; 6 is a flow chart showing an example of processing by a contact timing determination unit; FIG. 7 is a diagram illustrating a hardware configuration example of a determination device according to the second embodiment of the present disclosure; FIG. It is a block diagram which shows the structural example of a determination apparatus.

[First embodiment]
A first embodiment of the present disclosure will be described with reference to FIGS. 1 to 9. FIG. FIG. 1 is a diagram showing a configuration example of a determination system 100. As shown in FIG. FIG. 2 is a diagram for explaining an example of image data acquired by the determination system 100. As shown in FIG. FIG. 3 is a block diagram showing a configuration example of the determination device 300. As shown in FIG. FIG. 4 is a diagram showing an example of the skeleton information 323. As shown in FIG. FIG. 5 is a diagram showing an example of the contact determination information 324. As shown in FIG. FIG. 6 is a diagram for explaining an example of determination by the contact determination unit 333. In FIG. FIG. 7 is a diagram for explaining an example of determination by the touchdown timing determining section 334. In FIG. FIG. 8 is a flow chart showing an example of processing by the contact determination unit 333. As shown in FIG. FIG. 9 is a flow chart showing an example of processing by the contact timing determination section 334. As shown in FIG.

In the first embodiment of the present disclosure, a determination system 100 that determines the timing at which the foot of a walking person touches the ground based on image data acquired using an imaging device such as the smartphone 200 will be described. The determination system 100 acquires time-series image data showing how a person walks from the back side to the front side of the screen. As will be described later, the determination system 100 recognizes the position of the skeleton from each of the plurality of image data. Then, the determination system 100 determines the timing at which the foot touches the ground based on the recognized change in position.

It should be noted that the timing of landing determined by the determination system 100 can be, for example, an index such as the angle of each joint at the timing when the foot touches the floor, whether the toe is raised from the floor, pitch, stride, various steps such as the step cycle. It can be used for gait analysis. The touchdown timing determined by the determination system 100 may be used in addition to the above examples.

FIG. 1 shows a configuration example of the determination system 100. FIG. Referring to FIG. 1 , the determination system 100 has, for example, a smart phone 200 and a determination device 300 . As shown in FIG. 1, the smartphone 200 and the determination device 300 are connected, for example, wirelessly or by wire so that they can communicate with each other.

The smart phone 200 functions as an imaging device that captures how a person walks. The smartphone 200 may be a smartphone having general functions such as a camera function for acquiring image data, a touch panel for displaying a screen, and various sensors such as a GPS sensor and an acceleration sensor.

In the case of this embodiment, as shown in FIG. 2, the smartphone 200 shoots a person walking in the front-back direction, such as from the back of the screen to the front. In other words, smartphone 200 acquires time-series image data showing how a person walks in the front-to-back direction. The smartphone 200 also transmits the acquired image data to the determination device 300 . The smartphone 200 may associate information indicating the date and time when the image data was acquired with the image data, and transmit the associated data to the determination device 300 .

The determination device 300 is an information processing device that determines the timing at which the foot of a walking person touches the ground based on the image data acquired by the smartphone 200 . For example, the determination device 300 is a server device or the like. The determination device 300 may be a single information processing device, or may be implemented on a cloud, for example.

FIG. 3 shows a configuration example of the determination device 300. FIG. Referring to FIG. 3, the determination device 300 has, for example, a communication I/F section 310, a storage section 320, and an arithmetic processing section 330 as main components.

The communication I/F unit 310 consists of a data communication circuit. Communication I/F unit 310 performs data communication with an external device, smartphone 200, or the like connected via a communication line.

The storage unit 320 is a storage device such as a hard disk or memory. The storage unit 320 stores processing information and programs 326 necessary for various processes in the arithmetic processing unit 330 . The program 326 realizes various processing units by being read and executed by the arithmetic processing unit 330 . The program 326 is read in advance from an external device or recording medium via a data input/output function such as the communication I/F unit 310 and stored in the storage unit 320 . Main information stored in the storage unit 320 includes, for example, a learned model 321, image information 322, skeleton information 323, contact determination information 324, contact timing information 325, and the like.

The trained model 321 is a trained model used when the skeleton recognition unit 332 performs skeleton recognition. The learned model 321 is, for example, generated in advance by performing machine learning using teacher data such as image data containing skeletal coordinates in an external device, etc. It is acquired from a device or the like and stored in the storage unit 320 . Trained model 321 may be generated using known methods. The learned model 321 may be updated by re-learning processing using additional teacher data.

The image information 322 includes time-series image data acquired by the camera of the smartphone 200 . The image information 322 is generated and updated when the image acquisition unit 331 acquires image data.

For example, in the image information 322, identification information for identifying image data is associated with image data. The image information 322 may include information other than the above examples, such as information indicating the date and time when the smartphone 200 acquired the image data.

The skeleton information 323 includes information indicating the coordinates of each part of the person recognized by the skeleton recognition unit 332 (that is, indicating the position). The skeleton information 323 includes, for example, information indicating the coordinates of each part of each piece of time-series image data. For example, the skeleton information 323 is generated/updated as a result of processing by the skeleton recognition unit 332 .

FIG. 4 shows an example of the skeleton information 323. Referring to FIG. 4, in skeleton information 323, for example, identification information is associated with position information of each part. Here, the identification information is, for example, information indicating image data used when recognizing skeleton information. The identification information may correspond to identification information for identifying image data in the image information 322 . Also, the position information of each part includes information indicating the coordinates of each part in the image data, such as the position of the pelvis.

It should be noted that the part included in the position information of each part corresponds to the learned model 321 . For example, in FIG. 4, the pelvis, the center of the spine, ..., the right knee, the left knee, ..., the right ankle, the left ankle, ...
is exemplified. The position information of each part can include about 30 parts such as right shoulder, . . . , left elbow, . In the case of this embodiment, the position information of each part includes at least information indicating the coordinates of the right ankle and the left ankle. The parts included in the position information of each part may be other than those illustrated in FIG. 4 and the like.

The ground contact determination information 324 includes information indicating the determination result of the ground contact determination unit 333 . For example, the ground contact determination information 324 is generated and updated as a result of determination by the ground contact determination section 333 .

FIG. 5 shows an example of the contact determination information 324. FIG. As shown in FIG. 5, for example, in contact determination information 324, identification information and contact determination results are associated with each other. Here, the identification information is, for example, information indicating image data for which grounding determination has been performed. The identification information may correspond to identification information for identifying image data in the image information 322 . Further, the ground contact determination result indicates the ground contact determination result. For example, the ground contact determination result is "left foot" indicating the result of determining that the left foot is on the ground, "right foot" indicating the result of determining that the right foot is on the ground, and "determining that it is excluded from the ground contact determination". -" etc. are included. The contact determination result may indicate other than the above examples.

The touchdown timing information 325 includes information indicating the determination result of the touchdown timing determination unit 334 . For example, the touchdown timing information 325 is generated and updated as a result of determination by the touchdown timing determination section 334 .

For example, the ground contact timing information 325 includes information indicating the timing at which the foot touches the ground. Here, the information indicating the timing at which the foot touched the ground is, for example, at least one of identification information indicating the image data determined to have touched the ground, information indicating the acquisition date and time of the image data determined to have touched the ground, and the like. be. The contact timing information 325 may be, for example, information indicating the foot that has touched the ground and information indicating the timing at which the foot touched the ground in association with each other.

The arithmetic processing unit 330 has an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 330 reads the program 326 from the storage unit 320 and executes it, so that the hardware and the program 326 work together to realize various processing units. Main processing units realized by the arithmetic processing unit 330 include, for example, an image acquisition unit 331, a skeleton recognition unit 332, a contact determination unit 333, a contact timing determination unit 334, an output unit 335, and the like. .

The image acquisition unit 331 acquires image data acquired by the smartphone 200 from the smartphone 200 via the communication I/F unit 310 . For example, the image acquisition unit 331 acquires time-series image data from the smartphone 200 . The image acquisition unit 331 also stores the acquired image data in the storage unit 320 as the image information 322 .

The skeleton recognition unit 332 uses the trained model 321 to recognize the skeleton of the person whose walking posture is to be measured in the image data. For example, the skeleton recognition unit 332 recognizes the skeleton of each piece of time-series image data. The skeleton recognition unit 332 may be configured to recognize the skeleton of image data extracted based on a predetermined condition from time-series image data. For example, the skeleton recognition unit 332 recognizes each part such as the upper part of the spine, the right shoulder, the left shoulder, the right elbow, the left elbow, the right wrist, the left wrist, the right hand, the left hand, and so on. The skeleton recognition unit 332 also calculates the coordinates of the recognized parts in the screen data. Then, the skeleton recognition unit 332 associates the recognition/calculation results with identification information for identifying the image data and stores them in the storage unit 320 as the skeleton information 323 .

The parts recognized by the skeleton recognition unit 332 correspond to the trained model 321 (teaching data used when learning the trained model 321). Therefore, the skeleton recognition unit 332 may recognize parts other than those exemplified above according to the trained model 321 .

The grounding determination unit 333 determines which foot is in contact with the ground based on the skeleton of the person indicated by the skeleton information 323 . For example, the contact determination unit 333 determines which of the right and left legs of the walking person is on the ground, based on the degree of change in the coordinates of the right and left ankles included in the skeleton information 323. do. For example, the contact determination unit 333 obtains information indicating the coordinates of the ankle by referring to the skeleton information 323 . Then, when the degree of change in the coordinates of the ankle satisfies a predetermined condition, the ground contact determination unit 333 determines that the foot having the ankle that satisfies the condition is in contact with the ground. Further, the contact determination unit 333 stores information indicating the result of the decision in the storage unit 320 as contact determination information 324 .

FIG. 6 shows an example of changes in the Y coordinates (coordinates in the vertical direction of the image data) of the right and left ankles. For example, in FIG. 6, the X axis indicates values corresponding to time, and the Y axis indicates vertical coordinates (Y coordinates) in image data. For example, in the case of FIG. 6, the larger the Y-axis value, the lower the right or left ankle in the image data (that is, the closer the right or left ankle is to the front of the screen). It is shown that.

In general, when a person walks, one of the right and left feet remains on the ground and the other moves forward, both the right and left feet touch the ground, and one of the right and left feet remains on the ground and one moves forward. , and both right and left feet touch the ground, repeating this exercise alternately. That is, in the case of this embodiment, the Y coordinate of the right ankle is constant and the Y coordinate of the left ankle is increased, the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant, the Y coordinate of the left ankle is constant and the Y coordinate of the right ankle is constant. It is assumed that the state in which the coordinates increase and the Y coordinate of the right ankle and the Y coordinate of the left ankle are constant is repeated.

Therefore, the ground contact determination unit 333 determines which foot is grounded based on the degree of change in the Y coordinates of the right ankle and the left ankle. In other words, the ground contact determination unit 333 determines that the foot having the ankle is in contact with the ground when it can be evaluated that the Y coordinate of the ankle is constant. Specifically, for example, the contact determination unit 333 compares the Y coordinate of the ankle in the image data to be determined with the Y coordinate of the ankle in the image data preceding the determination target in the time-series image data. . Then, when the displacement (amount of change) of the ankle from the Y coordinate in the previous image data is equal to or less than a predetermined value, the contact determination unit 333 assumes that the Y coordinate is constant and determines the ankle with the constant Y coordinate. It is determined that the foot that has the ground is in contact with the ground. Note that the contact determination unit 333 may determine that the Y coordinate is constant by a method other than the above-described example. Further, the range of displacement in which the ground contact determination unit 333 determines that the Y coordinate is constant may be set arbitrarily.

Further, the contact determination unit 333 is configured to determine to exclude from contact determination when it is determined that the leg other than the leg determined to be in contact with the ground has started to move based on the Y-coordinate of the ankle. You may For example, in general walking, the left foot begins to move after the right foot touches the ground. After the Y coordinate of the right ankle becomes constant, the contact determination section 333 may make a decision to exclude from contact determination at the timing when the Y coordinate of the left ankle starts to increase. Specifically, for example, after the Y coordinate of the right ankle becomes constant, the contact determination unit 333 determines that the displacement of the Y coordinate of the left ankle from the previous image data in the time-series image data When the displacement of the Y coordinate of the right ankle from the image data is exceeded, it can be determined that the left foot has started to move and can be excluded from the ground contact determination.

Also, in general, when a person walks, the foot that newly touches the ground advances further than the foot that has already touched the ground. That is, in the case of this embodiment, the Y coordinate of the ankle of the newly grounded foot is larger than the Y coordinate of the ankle of the already grounded foot. Therefore, the contact determination unit 333 may be configured to determine whether or not the foot is contacting the ground by focusing on only the ankle having the larger Y coordinate between the right ankle and the left ankle.

Further, the contact determination unit 333 may be configured to refer to the coordinates of the knee in addition to the coordinates of the ankle when determining the contact. For example, the contact determination unit 333 may be configured to make a decision to exclude from the contact decision when the coordinate change of the knee satisfies a predetermined condition. Specifically, for example, the ground contact determination unit 333 determines whether the displacement of the Y coordinate of the knee from the previous image data in the time-series image data is 0 or less for the leg to be determined whether or not it is grounded. , it may be configured to make a determination to exclude it from the ground contact determination.

For example, as described above, the contact determination unit 333 makes contact determination based on the degree of change in the Y coordinate of the ankle. Note that the contact determination unit 333 may be configured to perform contact determination based on the degree of change in the skeleton other than the above example, such as using the Y coordinate of the toe portion.

The ground contact timing determination unit 334 determines the timing at which the foot touches the ground based on the ground contact state of the foot determined based on the skeleton of the person indicated by the skeleton information 323 . For example, the contact timing determination unit 334 determines the timing at which the foot contacts the ground based on the change in the contacting foot indicated by the contact determination information 324 . Then, the contact timing determination section 334 stores the decision result as the contact timing information 325 in the storage section 320 .

For example, the ground contact timing determination unit 334 determines that the foot has touched the ground at the timing when the grounded foot indicated by the ground contact determination result is switched to another foot. For example, referring to FIG. 7, the "left foot" is grounded in the identification information "124" and "125", and is excluded from the grounding determination in the identification information "126" and "127". After that, it is determined that the "right foot" is on the ground with the identification information "128". That is, in the case of FIG. 7, the foot on the ground is switched from the left foot to the right foot at the timing of the identification information "128". Therefore, the ground contact timing determination unit 334 determines that the right foot has grounded at the timing of the identification information "128".

The output unit 335 outputs contact timing information 325, contact determination information 324, and the like. For example, the output unit 335 can output at least one of the above information to an external device, the smart phone 200, or the like.

The above is a configuration example of the determination device 300 . Next, the operation of the determination device 300 will be described with reference to FIGS. 8 and 9. FIG.

First, with reference to FIG. 8, an operation example of the contact determination unit 333 will be described. Referring to FIG. 8, the contact determination unit 333 compares the Y coordinate of the knee in the image data to be determined with the Y coordinate of the knee in the image data preceding the determination target in the time-series image data ( step S101). If the displacement of the knee from the Y coordinate in the previous image data exceeds 0 (step S101, Yes), the contact determination unit 333 confirms whether or not the displacement of the ankle satisfies the condition (step S102). .

When the displacement of the ankle satisfies the condition (step S102, Yes), the ground contact determination unit 333 determines that the foot satisfying the condition is in contact with the ground (step S103). On the other hand, if the displacement of the Y coordinate of the knee is 0 or less (step S101, No) or if the displacement of the ankle does not satisfy the condition (step S102, No), the ground contact determination unit 333 determines that the foot that does not satisfy the condition is grounded. decide not to.

The above is an operation example of the contact determination unit 333 . For example, when the Y-coordinate of the ankle can be evaluated to be constant based on the degree of change in the Y-coordinate of the ankle, the contact determination unit 333 determines that the leg other than the leg determined to be on the ground starts to move. If no determination is made or if the coordinate change of the knee satisfies the condition, it can be determined that the displacement of the ankle satisfies the condition. Further, the contact determination unit 333 may be configured to identify the ankle of interest based on the Y-coordinate of the ankle and perform the above-described confirmation on the identified ankle.

Next, with reference to FIG. 9, an operation example of the ground contact timing determination section 334 will be described. Referring to FIG. 9, the contact timing determination unit 334 refers to the contact timing information 325 and checks whether or not the contacting foot indicated by the contact decision result has changed (step S201).

When the foot has changed (step S201, Yes), the contact timing determination unit 334 determines that the changed timing is the contact timing (step S202). On the other hand, if the grounded foot indicated by the determination result has not changed (step S201), the ground contact timing determining section 334 does not perform the above determination.

The above is a processing example of the contact timing determination unit 334 .

As described above, the determination device 300 has the skeleton recognition section 332 , the contact determination section 333 and the contact timing determination section 334 . According to such a configuration, the contact timing determination unit 334 determines the timing at which the foot touches the ground based on the determination result of the contacting foot made by the contact determination unit 333 using the coordinates of the skeleton recognized by the skeleton recognition unit 332 . can judge. That is, according to the above configuration, it is possible to grasp the timing when the foot touches the ground based on the image data.

In addition, the determination device 300 determines the contact timing as a result of focusing on the part near the foot that is actually in contact with the ground. Therefore, the determination device 300 can perform determination closer to the actual condition than the method using the displacement of the thoracolumbar region.

Note that the determination device 300 may be configured to determine the contact timing based on the result of skeleton recognition by an external device that recognizes the skeleton. When configured in this manner, determination device 300 does not need to have the function of skeleton recognition section 332 .

[Second embodiment]
Next, a second embodiment of the present disclosure will be described with reference to FIGS. 10 and 11. FIG. In a second embodiment of the present disclosure, an overview of the configuration of a determination device 400, which is an information processing device, will be described.

FIG. 10 shows a hardware configuration example of the determination device 400 . Referring to FIG. 10, the determination device 400 has the following hardware configuration as an example.
- CPU (Central Processing Unit) 401 (arithmetic unit)
・ROM (Read Only Memory) 402 (storage device)
・RAM (Random Access Memory) 403 (storage device)
Program group 404 loaded into RAM 403
- Storage device 405 for storing program group 404
- A drive device 406 that reads and writes a recording medium 410 outside the information processing device
- A communication interface 407 that connects to a communication network 411 outside the information processing apparatus
An input/output interface 408 for inputting/outputting data
A bus 409 connecting each component

Also, the determination device 400 can realize the function of the determination unit 421 shown in FIG. The program group 404 is stored in the storage device 405 or the ROM 402 in advance, for example, and is loaded into the RAM 403 or the like and executed by the CPU 401 as necessary. The program group 404 may be supplied to the CPU 401 via the communication network 411 or stored in the recording medium 410 in advance, and the drive device 406 may read the program and supply it to the CPU 401 .

Note that FIG. 10 shows a hardware configuration example of the determination device 400 . The hardware configuration of the determination device 400 is not limited to the case described above. For example, the determination device 400 may be composed of part of the configuration described above, such as not having the drive device 406 .

The determination unit 421 acquires information indicating the position where the predetermined part exists, which is acquired from each time-series image data, and determines whether the foot touches the ground based on the degree of change in the position where the acquired part exists. determine the timing.

As described above, the determination device 400 has the determination unit 421 . With such a configuration, the determination unit 421 can determine the timing at which the foot touches the ground based on the degree of change in the position of the part. This makes it possible to grasp the timing at which the foot touches the ground based on the image data.

An information processing device such as the determination device 400 described above can be realized by installing a predetermined program in the information processing device. Specifically, a program according to another aspect of the present invention acquires, in an information processing apparatus, information indicating a position where a predetermined site exists, which is acquired from each time-series image data, and acquires A program for realizing a process of determining the timing at which the foot touches the ground based on the degree of change in the position where the part exists.

Further, in the determination method executed by the information processing device described above, the information processing device acquires information indicating the position where the predetermined part exists, which is acquired from each time-series image data, and acquires The timing at which the foot touches the ground is determined based on the degree of change in the position of the part.

Even an invention such as a program (or a recording medium) or a determination method having the above-described configuration can achieve the above-described object of the present invention because it has the same actions and effects as the above-described case. I can.

<Appendix>
Some or all of the above embodiments may also be described as the following appendices. The outline of the determination device and the like according to the present invention will be described below. However, the present invention is not limited to the following configurations.

(Appendix 1)
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. A determination device having a determination unit.
(Appendix 2)
having a judgment unit that judges the foot that is in contact with the ground based on the degree of change in the position where the part exists;
The determination device according to supplementary note 1, wherein the determination unit determines the timing at which the foot touches the ground based on the determination result of the determination unit.
(Appendix 3)
The determination device according to appendix 2, wherein the determination unit determines that the foot has touched the ground at a timing at which it is determined that the grounded foot has changed, based on the determination result of the determination unit.
(Appendix 4)
The information indicating the position where the part exists includes information indicating the position where the ankle exists, which is determined by recognizing the skeleton of the person in the image data,
The determination device according to appendix 2 or appendix 3, wherein the judging unit judges which foot is in contact with the ground based on how the position of the ankle changes.
(Appendix 5)
The determination device according to appendix 4, wherein the determination unit determines that the foot having the ankle is in contact with the ground when it can be evaluated that the position of the ankle is constant based on the degree of change in the position of the ankle.
(Appendix 6)
When it is determined that the leg other than the leg determined to be on the ground based on the degree of change in the position of the ankle has started to move, the determination unit determines to exclude it from the grounding determination. 5. The determination device according to 5.
(Appendix 7)
The determination unit according to any one of appendices 4 to 6, wherein the determination unit determines the target foot based on the position of the ankle, and determines whether or not the target foot is in contact with the ground. judgment device.
(Appendix 8)
The information indicating the position where the part exists includes information indicating the position where the knee exists, which is determined by recognizing the skeleton of the person in the image data,
7. The determination device according to any one of appendices 4 to 7, wherein the judgment unit judges which foot is in contact with the ground based on the degree of change in the position of the ankle and the degree of change in the position of the knee.
(Appendix 9)
The information processing device
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. judgment method.
(Appendix 10)
information processing equipment,
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. A program for realizing processing.

The programs described in each of the above embodiments and supplementary notes are stored in a storage device or recorded in a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Although the present invention has been described with reference to the above-described embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

In addition, the present invention enjoys the benefit of the priority claim based on the patent application of Japanese Patent Application No. 2021-055312 filed on March 29, 2021 in Japan, and is described in the patent application. The contents are hereby incorporated by reference in their entirety.

100 determination system 200 smart phone 300 determination device 310 communication I/F unit 320 storage unit 321 trained model 322 image information 323 skeleton information 324 contact determination information 325 contact timing information 326 program 330 arithmetic processing unit 331 image acquisition unit 332 skeleton recognition unit 333 Contact determination unit 334 Contact timing determination unit 335 Output unit 400 Determination device 401 CPU
402 ROMs
403 RAM
404 program group 405 storage device 406 drive device 407 communication interface 408 input/output interface 409 bus 410 recording medium 411 communication network 421 determination unit

Claims

Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. A determination device having a determination unit.
having a judgment unit that judges the foot that is in contact with the ground based on the degree of change in the position where the part exists;
The determination device according to claim 1, wherein the determination unit determines timing when the foot touches the ground based on the result of determination by the determination unit.
The determination device according to claim 2, wherein the determination unit determines that the foot has touched the ground at a timing at which it is determined that the grounded foot has changed, based on the determination result of the determination unit.
The information indicating the position where the part exists includes information indicating the position where the ankle exists, which is determined by recognizing the skeleton of the person in the image data,
The determination device according to claim 2 or 3, wherein the determining unit determines which foot is in contact with the ground based on how the position of the ankle changes.
5. The determination device according to claim 4, wherein the determination unit determines that the foot having the ankle is in contact with the ground when it can be evaluated that the position of the ankle is constant based on the degree of change in the position of the ankle.
5. When it is determined that the leg other than the leg determined to be on the ground based on the degree of change in the position of the ankle has started to move, the determination unit determines to exclude it from the grounding determination. The determination device according to claim 5.
7. The determination unit determines whether or not the target foot is in contact with the ground based on the position of the ankle. Determination device as described.
The information indicating the position where the part exists includes information indicating the position where the knee exists, which is determined by recognizing the skeleton of the person in the image data,
8. The determination device according to any one of claims 4 to 7, wherein the determining unit determines which foot is in contact with the ground based on the degree of change in the position of the ankle and the degree of change in the position of the knee.
The information processing device
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. judgment method.
information processing equipment,
Information indicating the position where a predetermined part exists is acquired from each time-series image data, and the timing at which the foot touches the ground is determined based on the degree of change in the position where the acquired part exists. A computer-readable recording medium in which a program for realizing processing is recorded.