JP2023085658A - Feature-point position estimation device, feature-point position estimation method and program - Google Patents

Abstract

To estimate a position of a body feature point.SOLUTION: A feature-point position estimation device for estimating three-dimensional coordinates of a prescribed feature point of a body located at a target segment of an object, includes: a position input part for inputting three-dimensional coordinate values of a first body feature point for identifying a position in a first side of the target segment of the object and of a second body feature point for identifying a position in a second side, and three-dimensional coordinate values of a plurality of body surface points of the object; a plane calculation part for calculating a plane in a three-dimensional space passing through the first body feature point, the plane perpendicular to a straight line connecting the first body feature point and the second body feature point; an extraction part for extracting a plurality of body surface points on the plane, a distance of which from the plane is a prescribed range out of the plurality of body surface points; an approximation part for approximating the plurality of body surface points on the plane to a space ellipse on a three-dimensional space; a condition input part for inputting conditions representing at which point on the space ellipse the prescribed body feature point is located; and a position estimation part for estimating the three-dimensional coordinate values of the prescribed body feature point on the basis of, the space ellipse and conditions.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to techniques for estimating the positions of body feature points.

Conventionally, in rehabilitation medicine, orthopedic treatment, etc., three-dimensional motion analysis using motion capture has been performed in order to measure and analyze the motion of a subject. A motion analysis method using markerless motion capture is known as one of the methods for evaluating motion in three-dimensional motion analysis. As markerless motion capture, for example, a distance image sensor (depth sensor) is used to measure the distance to the target by irradiating the target with an infrared laser, and the TOF (Time of Flight) method is used to obtain information on the distance to the target. There is known a method of acquiring a depth image (distance image) having , and deriving (estimating) the joint positions of a target from the acquired depth image (Non-Patent Document 1). Also, Non-Patent Document 2 describes a method of estimating joint positions of a target from depth images using Kinect (registered trademark) as a motion capture device.

Unlike optical motion capture and inertial motion capture, markerless motion capture does not require attachment of markers or sensors to objects such as people or objects. Therefore, there is no need to perform complicated measurement preparations, and since it is possible to measure the subject's natural movements, markerless motion capture is used as a highly useful technique in various situations. For example, a method of estimating skeletal information based on distance images using markerless motion capture has been proposed (Patent Document 1), and the use of markerless motion capture in clinical settings is also expected (non-patent Reference 3).

Patent No. 6617830

Takashi Nishibayashi, “Mechanism of Kinect and Natural User Interface”, Journal of the Institute of Image Information and Television Engineers Vol. 66 No. 9 2012, p. 755-759 “Azure Kinect DK Document”, [online], Microsoft, [searched on October 27, 2021], Internet <URL: https://docs.microsoft.com/ja-jp/azure/Kinect-dk/ ＞ Koichi Haruna, 3 others, "Application example of 3D motion analysis by markerless motion capture", Journal of Japanese Society of Prosthetics and Orthotics Vol. 35 No. 1 2019, p. 17-23

As described above, in the motion analysis method using markerless motion capture, a depth image is acquired by a depth image sensor, and the positions (three-dimensional position coordinates) of body feature points such as joints of the object (subject) are acquired. Is possible.

However, while the distance image sensor can acquire (output) the position coordinates of predetermined body feature points such as joints and bones in the distance image sensor, there is no predetermined position coordinate in the distance image sensor. Obtaining locations for body feature points is difficult. Therefore, there is a problem that the desired joint angle cannot be calculated only with the three-dimensional position coordinates of the body feature points determined in the range image sensor.

For example, in the case of the lower limbs, the range image sensor normally outputs three-dimensional position coordinates of knee joints, ankle joints, and toes, which are predetermined body feature points in the range image sensor. According to the three-dimensional positional coordinates of these body feature points, it is possible to calculate the angles of plantar flexion/dorsiflexion and the angles of adduction/abduction of the ankle joint using projection angles. It is difficult to correctly calculate the varus/valgus angle. In order to calculate the varus/valgus angle of the ankle joint, it is possible to use, for example, Euler angles between body segments (between body segment coordinate systems). However, in a body segment where only the position coordinates of two body feature points are output, such as the foot, where only the three-dimensional position coordinates of the ankle joint and the toe are output, the coordinate system cannot be generated (identified), and the Euler angles ( varus-valgus angle) is difficult to calculate.

An object of the present disclosure is to estimate the positions of body feature points in view of the above problems.

An example of this disclosure is:
A feature point position estimation device for estimating a three-dimensional coordinate value of a predetermined body feature point located in a target segment of an object,
three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment of the object and a second body feature point specifying the position of the second side of the body segment of the object; position input means for inputting three-dimensional coordinate values of body surface points;
A plane in a three-dimensional space passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is defined as the first body feature point and the second body feature point. Plane calculation means for calculating using three-dimensional coordinate values of body feature points;
extracting means for extracting a plurality of planar body surface points whose distance from the plane is within a predetermined range from the plurality of body surface points;
approximation means for approximating the plurality of planar body surface points to a spatial ellipse, which is an ellipse in a three-dimensional space, using the three-dimensional coordinate values of the plurality of planar body surface points;
condition input means for inputting a condition indicating at which position on the spatial ellipse the predetermined body feature point exists;
position estimating means for estimating three-dimensional coordinate values of the predetermined body feature points based on the spatial ellipse and the conditions;
It is a feature point position estimation device comprising: Since a body surface such as human limbs can be approximated to an ellipse, in the present disclosure, the body surface (a collection of body surface points) is approximated by an ellipse.

In one aspect, the approximation means obtains an equation capable of obtaining three-dimensional coordinate values of points on the spatial ellipse.

In one aspect, the approximation means is an equation of a two-dimensional space ellipse that matches the space ellipse by moving on the three-dimensional space as an equation capable of acquiring three-dimensional coordinate values of points on the space ellipse. Then, a transformation equation for transforming a point on the ellipse in the two-dimensional space to a point on the space ellipse is obtained.

In one aspect, the position estimating means inputs the coordinate values on the ellipse in the two-dimensional space corresponding to the input condition into the transformation equation, thereby obtaining the three-dimensional coordinate values of the predetermined body feature point. to estimate

In one aspect, the condition is such that the shape of the body surface of the target body segment on a plane passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is an ellipse. This is a condition determined based on the position of the third body feature point on the ellipse when it is assumed that the approximation is performed by .

In one aspect, the condition is a condition indicating that the predetermined body feature point is any vertex of the spatial ellipse.

In one aspect, the three-dimensional coordinate values of the first body feature point and the second body feature point are obtained by a three-dimensional measuring device that measures the three-dimensional coordinate values of predetermined body feature points of an object. are coordinate values.

In one aspect, using the three-dimensional coordinate values of the first body feature point and the second body feature point and the estimated three-dimensional coordinate value of the predetermined body feature point, It further comprises coordinate system generating means for generating a coordinate system.

In one aspect, the apparatus further comprises angle calculation means for calculating the joint angle associated with the movement of the target body segment by calculating the Euler angle between the Cartesian coordinate system of the target body segment and another Cartesian coordinate system. .

In one aspect, the body segment of the subject is one of a plurality of body segments of the subject;
Each of the position input means, the plane calculation means, the approximation means, the condition input means, the feature point position estimation means, and the coordinate system generation means, in addition to the target body segment, performing processing on other body segments different from the body segment of the subject;
The angle calculation means calculates a joint angle associated with the movement of the target body segment by calculating an Euler angle between the orthogonal coordinate system of the target body segment and the orthogonal coordinate system of the other body segment.

In one aspect, the segment is the foot,
the first body feature point is a toe;
the second body feature point is an ankle joint;
The predetermined physical feature point is the widthwise end of the foot on the toe side.

In one aspect, the predetermined body landmark is the fifth metatarsal,
The condition is a condition indicating that the fifth metatarsal bone, which is the predetermined physical feature point, is the vertex in the lateral direction of the major axis of the spatial ellipse;
The position estimation means estimates a three-dimensional coordinate value of the fifth metatarsal bone, which is the predetermined physical feature point.

The present invention
A computer for estimating three-dimensional coordinate values of predetermined body feature points located in target body segments of an object,
three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment of the object and a second body feature point specifying the position of the second side of the body segment of the object; a position input step of inputting three-dimensional coordinate values of body surface points;
A plane in a three-dimensional space passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is defined as the first body feature point and the second body feature point. a plane calculation step of calculating using the three-dimensional coordinate values of the body feature points;
an extracting step of extracting a plurality of planar body surface points whose distance from the plane is within a predetermined range from the plurality of body surface points;
an approximation step of approximating the plurality of planar body surface points to a spatial ellipse, which is an ellipse in a three-dimensional space, using the three-dimensional coordinate values of the plurality of planar body surface points;
a condition input step of inputting a condition indicating at which position on the spatial ellipse the predetermined body feature point exists;
a position estimation step of estimating a three-dimensional coordinate value of the predetermined body feature point based on the spatial ellipse and the condition;
can be defined as a feature point location estimation method that performs

The present invention
a computer for estimating three-dimensional coordinate values of predetermined body feature points located on target body segments of an object;
three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment of the object and a second body feature point specifying the position of the second side of the body segment of the object; position input means for inputting three-dimensional coordinate values of body surface points;
A plane in a three-dimensional space passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is defined as the first body feature point and the second body feature point. Plane calculation means for calculating using three-dimensional coordinate values of body feature points;
extracting means for extracting a plurality of planar body surface points whose distance from the plane is within a predetermined range from the plurality of body surface points;
approximation means for approximating the plurality of planar body surface points to a spatial ellipse, which is an ellipse in a three-dimensional space, using the three-dimensional coordinate values of the plurality of planar body surface points;
condition input means for inputting a condition indicating at which position on the spatial ellipse the predetermined body feature point exists;
position estimating means for estimating three-dimensional coordinate values of the predetermined body feature points based on the spatial ellipse and the conditions;
can be defined as a program for functioning as

The present disclosure can be understood as an apparatus, a system, a computer-executed method, or a computer-executed program. The present disclosure can also be understood as recording such a program in a recording medium readable by a computer, other device, machine, or the like. Here, a computer-readable recording medium is a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc. say.

According to the present disclosure, it is possible to estimate the positions of body feature points.

1 is a schematic diagram showing the configuration of a system according to an embodiment; FIG. It is a figure which shows the outline of a functional structure of the feature point position estimation apparatus which concerns on embodiment. It is a figure which shows an example of the leg|foot part which concerns on embodiment. It is a figure which shows an example of joint angle calculation which concerns on embodiment. 4 is a flowchart showing an overview of the flow of feature point position estimation processing according to the embodiment; 6 is a flowchart showing an overview of the flow of angle calculation processing according to the embodiment;

Hereinafter, embodiments of an apparatus, a method, and a program according to the present disclosure will be described based on the drawings. However, the embodiment described below is an example of the embodiment, and does not limit the device, method, and program according to the present disclosure to the specific configurations described below. For implementation, a specific configuration may be appropriately adopted according to the mode of implementation, and various improvements and modifications may be made.

In this embodiment, an embodiment will be described in which the device, method, and program according to the present disclosure are implemented in a feature point position estimation device that estimates the positions of body feature points on the feet of a subject. However, the device, method, and program according to the present disclosure can be widely used for techniques for estimating the position of any body feature point of an object, and the application target of the present disclosure is shown in the embodiment Examples are not limiting.

<System configuration>
FIG. 1 is a schematic diagram showing the configuration of a system according to this embodiment. As shown in FIG. 1, a system 9 according to the present embodiment includes coordinate data (two-dimensional image coordinates and depth information) of body surface points (observation points) of an object and coordinates of positions of body feature points such as joints. A three-dimensional measuring device 3 that acquires (measures) data (three-dimensional coordinates) and the coordinate data acquired by the three-dimensional measuring device 3 are recorded. and a feature point position estimating device 1 for estimating the positions of points). Here, the object is not limited to a person such as a subject, and may be any animal, robot, or the like as long as it has a physical characteristic point such as a bone or a joint. In this embodiment, a case where the subject is a subject (person) will be exemplified. Note that the body feature points in the present embodiment refer to feature points related to the skeleton of an object such as bones and joints, and are exemplified by the head, shoulders, trunk, hips (pelvis), fingers, toes, joints of limbs, and the like. be.

The three-dimensional measuring device 3 is a distance image sensor that acquires a distance image (depth image), which is an image having distance information to an object (subject) (an image having distance information in the depth direction for each pixel in the image). (depth sensor) and a position acquisition unit that acquires the position (three-dimensional coordinate value) of a predetermined physical feature point (joint or the like) of the object (subject). Note that the three-dimensional measuring device 3 may include a video camera (color camera (RGB camera)) that acquires a color image (RGB image) of the object (subject). For example, the three-dimensional measuring device 3 is exemplified by Kinect (registered trademark), which is a motion capture device equipped with a distance image sensor and a video camera. However, the three-dimensional measuring device 3 is not limited to Kinect, and may be any device as long as it acquires the positions of body feature points such as joints. Further, the specific hardware configuration of the three-dimensional measurement device 3 can be appropriately omitted, replaced, or added according to the mode of implementation.

A range image sensor is a sensor that can measure the distance (depth) to an observation point on an object, and is mainly equipped with a projector that emits infrared rays and a camera (infrared camera) that receives the reflected infrared rays. . Note that the observation point of the object in this embodiment is a point on the surface (body surface) of the object whose distance is measured by the distance image sensor, and each pixel of the distance image acquired by the distance image sensor This is a corresponding point. A distance image sensor mainly measures the distance to an object (observation point) by a TOF method or a pattern irradiation method. The TOF method is based on the time it takes for the irradiated (projected) infrared laser to make a round trip to the object (observation point) (the time from irradiation to receiving the reflected light), and the distance to the object (observation point). This is a method for measuring distance. The pattern irradiation method measures the distance to an object (observation point) by irradiating an infrared laser with a specific pattern and analyzing the distortion of the reflected light pattern. In this embodiment, any method may be used as the method for the distance image sensor to measure the distance to the object.

A range image sensor measures the distance to an object and converts the pixel value of each pixel in the image into the distance information (depth value ) is acquired (captured). In the range image, for each observation point, two-dimensional image coordinate values (position coordinates in the range image) (u, v) in the uv orthogonal coordinate system and the two-dimensional image coordinate values (u, v) The distance (depth value) d from the corresponding range image sensor is included. The distance d is the distance between the range image sensor and the object. Hereinafter, (u, v, d) will be referred to as "two-dimensional coordinate values" for each observation point (pixel).

The position acquisition unit obtains the position (three-dimensional coordinate value (X _c , Y _c , Z _c )). The position acquisition unit obtains, for example, a plurality of body feature points (for example, 25 body feature points) such as head, shoulders, trunk, hips (pelvis), fingertips, toes, joints of limbs, etc., which are predetermined body feature points. 3D coordinate values of feature points). In this embodiment, in order to estimate body feature points on the foot, the position acquisition unit acquires the three-dimensional coordinate values of at least the ankle joint and the toe (near the second metatarsal bone). In the present embodiment, an example is shown in which the three-dimensional measuring device 3 (position acquisition unit) acquires the three-dimensional coordinate values of the toe with the vicinity of the head of the second metatarsal bone as the toe. The 3D coordinate values of the toe may be acquired by using a site other than the vicinity (for example, the vicinity of the third metatarsal bone) as the toe.

Various arbitrary methods may be used as a method for acquiring the positions of predetermined body feature points in the three-dimensional measuring device 3 based on the distance image. For example, a method disclosed in Patent Document 1, a method of estimating by inputting distance information of a target object to a trained model (classifier), or the like may be used. Further, the three-dimensional coordinate values (X _c , Y _c , Z _c ) acquired by the position acquisition unit are in a three-dimensional orthogonal coordinate system (three-dimensional coordinate system of the range image sensor) with the range image sensor (infrared camera) as the origin. ).

If the three-dimensional measuring device 3 is equipped with a video camera that acquires an RGB image of the object, the position of the body feature point acquired by the position acquiring unit may be drawn on the RGB image. However, the range image sensor (infrared camera) and video camera are each associated with an independent two-dimensional coordinate system. In other words, the range image is defined by the two-dimensional coordinate system of the range image (range image sensor), and the RGB image is defined by the two-dimensional coordinate system of the RGB image (video camera). The same applies to the three-dimensional coordinate system. Therefore, when the three-dimensional coordinate values acquired by the position acquiring unit are drawn on an RGB image, the three-dimensional coordinate values of the positions of the body feature points are obtained by using a conversion function capable of converting coordinates between coordinate systems. Conversion to coordinate values on an RGB image is required. The exchange of coordinate systems is also described in Non-Patent Document 2, and any coordinate transformation may be performed.

From this, the system 9 according to the present embodiment is provided with the three-dimensional measuring device 3 described above, so that the two-dimensional coordinate values (u, v, d) and three-dimensional coordinate values (X _c , Y _c , Z _c ) for the positions of predetermined body feature points of the object (subject) are acquired.

Various calculations in the three-dimensional measuring device 3 are executed by a computer including a processor, memory, and the like. For example, the three-dimensional measuring device 3 includes an input unit that captures information acquired by a camera, a sensor, etc., a storage unit that stores information captured by the input unit and information calculated by the processing unit, and a storage unit that stores information captured by the input unit. A processing unit and the like are provided for performing various types of processing on information. Each function of these functional units (input unit, storage unit, processing unit) is executed by a general-purpose processor, but part or all of these functions may be executed by one or more dedicated processors. Also, part or all of these functions may be executed by a remotely installed device or a plurality of distributed devices using cloud technology or the like.

The feature point position estimation device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and an EEPROM (Electrically Erasable and Programmable Read Only Memory). , HDD (Hard Disk Drive), etc. , a communication device 15 such as a NIC (Network Interface Card), an input device 16 such as a keyboard, and an output device 17 such as a display. However, the specific hardware configuration of the feature point position estimation device 1 can be appropriately omitted, replaced, or added according to the mode of implementation. Moreover, the feature point position estimation device 1 is not limited to a device consisting of a single housing. The feature point position estimation device 1 may be realized by a plurality of devices using so-called cloud or distributed computing technology.

The feature point position estimation device 1 is the coordinate position of the observation point on the surface (body surface) of the object (subject) and the coordinates of the predetermined body feature points of the object, which are acquired by the three-dimensional measurement device 3. A device for estimating the position of a desired (predetermined) body feature point based on the position. In this embodiment, in the feature point position estimation device 1, based on the coordinate positions of the ankle joints and toes of the subject and the coordinate positions of a plurality of body surface points (observation points) of the foot, A case of estimating the position of a bone is illustrated. In this embodiment, the feature point position estimation device 1 and the three-dimensional measurement device 3, which are separate devices (separate housings), are exemplified. and the functions of the three-dimensional measuring device 3 may be provided.

FIG. 2 is a diagram showing an outline of the functional configuration of the feature point position estimation device according to this embodiment. In the feature point position estimation device 1, a program recorded in the storage device 14 is read out to the RAM 13, executed by the CPU 11, and each hardware provided in the feature point position estimation device 1 is controlled. , a position input unit 21 , a storage unit 22 , a condition input unit 23 , a plane calculation unit 24 , an extraction unit 25 , an approximation unit 26 , a position estimation unit 27 , a coordinate system generation unit 28 and an angle calculation unit 29 . In addition, in this embodiment and other embodiments described later, each function provided in the feature point position estimation device 1 is executed by the CPU 11, which is a general-purpose processor. may be executed by a dedicated processor of In addition, each functional unit provided in the feature point position estimation device 1 is not limited to being implemented in a device consisting of a single housing (one device), but is remotely and/or distributed (for example, cloud above) may be implemented.

The position input unit 21 inputs coordinate data of a plurality of observation points on the subject's body surface and predetermined (set) body feature points in the three-dimensional measuring device 3 (distance image sensor) of the subject. do. Specifically, the position input unit 21 inputs the three-dimensional coordinate values of a plurality of body surface points (observation points) of the subject and the target of the subject among the predetermined body feature points in the three-dimensional measuring device 3. Three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment and a second body feature point specifying the position of the second side of the body segment are entered. Here, the human body mainly includes a plurality of parts such as the head, trunk (upper trunk, middle trunk, lower trunk), upper arm, forearm, hand, thigh, lower leg, and foot. There are joints between the body segments that connect the body segments. Hereinafter, the segment where the (desired) body feature point whose position is to be estimated (hereinafter referred to as the "third body feature point") is located is referred to as the "target body segment".

The position input unit 21 first acquires, from the three-dimensional measuring device 3, two-dimensional coordinate values (u, v, d) of at least a plurality of body surface points of the subject in the target segment. In the present embodiment, the position input unit 21 acquires two-dimensional coordinate values for a plurality of body surface points of the subject on the foot (target segment) where the fifth metatarsal (third body feature point) is located. do. Then, the position input unit 21 converts the two-dimensional coordinate values (u, v, d) of the subject's body surface points to the three-dimensional coordinates of the distance image sensor by using a conversion function stored in the storage unit 22, which will be described later. Convert to coordinate system coordinate values (three-dimensional coordinate values (X _c , Y _c , Z _c )). Accordingly, the position input unit 21 acquires (inputs) three-dimensional coordinate values of at least a plurality of body surface points of the target body segment among the body surface points of the subject. Note that any method may be used as a method for converting from two-dimensional coordinate values to three-dimensional coordinate values, and for example, the DLT method (Direct Linear Transformation method) may be used.

In addition, the position input unit 21 receives from the three-dimensional measuring device 3 a first body feature point that specifies the position of the first side of the body segment of the object (subject) and a second body feature point that specifies the position of the second side. Input is performed by acquiring three-dimensional coordinate values (X _c , Y _c , Z _c ) for the positions of body feature points. In this embodiment, the locating input 21 is at least the position of the toe and foot that specifies the position of the first side of the foot (target segment) where the fifth metatarsal (third body feature point) is located. A case of inputting the three-dimensional coordinate values of the ankle joint specifying the position of the second side will be exemplified. A body feature point that is closer to the third body feature point is set as the first body feature point, and a body feature point that is not closer is set as the second body feature point. Therefore, in this embodiment, the first body feature point is the toe close to the fifth metatarsal, and the second body feature point is the ankle joint. Hereinafter, let the three-dimensional coordinate value of the first body feature point input by the position input unit 21 be P ₁ ((P ₁ =(p _1x , p _1y , p _1z ))), and the three-dimensional coordinate value of the second body feature point Let the coordinate value be P ₂ ((P ₂ =(p _2x , p _2y , p _2z ))).

In this embodiment, the position input unit 21 acquires coordinate data from the three-dimensional measuring device 3 to input the three-dimensional coordinate values of each point. The coordinate data is not limited to the example described above, and the coordinate data may be acquired from the recording device 14 or an external recording medium (not shown) that stores coordinate data measured (acquired) in advance. In addition, the position input unit 21 does not obtain the three-dimensional coordinate values of all predetermined body feature points in the distance image sensor, but instead obtains the three-dimensional coordinate values of the first body feature point and the second body feature point. Alternatively, only coordinate values may be obtained.

The storage unit 22 stores conversion functions. The conversion function transforms the two-dimensional coordinate values (u, v, d) obtained by the range image sensor into three-dimensional coordinate values ( X _c , Y _c , Z _c ). The storage unit 22 stores, for example, a conversion function that is used in the three-dimensional measuring device 3 (distance image sensor) to acquire the three-dimensional coordinate values of the positions of predetermined body feature points such as joints.

The condition input unit 23 inputs feature point determination conditions used to estimate (determine) the positions of the third body feature points. The feature point determination condition is the position of the third body feature point located on the body segment of interest on a spatial ellipse (approximate ellipse in three-dimensional space) that approximates the body surface point of the body segment of interest. It is a condition that indicates (specifies) whether For example, the feature point determination condition specifies that the third body feature point is one of the four vertices of the spatial ellipse (two vertices on the major axis side and two vertices on the minor axis side). It is a condition that However, the position of the third body feature point on the spatial ellipse specified in the feature point determination condition is not limited to the vertex of the spatial ellipse, and may be a point on the spatial ellipse other than the vertex. As described above, the body surface of human limbs and the like can be approximated to an ellipse, so in this embodiment, body surface points are approximated by an ellipse. Details of the spatial ellipse approximating the body surface points of the target segment will be described later.

The feature point determination condition is determined by the shape of the target segment (body surface shape) where the third body feature point is located. For example, the feature point determination condition is that the shape (cross-sectional shape) of the body surface of the target segment on a plane perpendicular to a straight line passing through the first body feature point and connecting the first body feature point and the second body feature point is represented by an ellipse. Assuming approximation, the position of the third body feature point on the ellipse is determined by a user such as a worker who performs motion analysis. Then, the user inputs the determined feature point determination condition to the feature point position estimation device 1, so that the condition input unit 23 can acquire the feature point determination condition.

For example, if the third body characteristic point is the fifth metatarsal bone, it is perpendicular to the straight line that passes through the first body characteristic point (toe) and connects the first body characteristic point (toe) and the second body characteristic point (ankle joint). Judgment (discrimination) that the shape of the body surface of the target body segment (foot) on a flat plane resembles a horizontally long oval shape (horizontal arch shape) when the foot is viewed from the front (toe side). is possible. Therefore, the characteristic point determination condition for the fifth metatarsal is determined (set) as the vertex of the major axis of the spatial ellipse (the vertex in the lateral direction of the body). Specifically, the feature point determination condition for the fifth metatarsal bone of the right foot includes the left-right axis in the three-dimensional Cartesian coordinate system (see FIG. 4) of the target body segment, which will be described later, among the vertices of the long axis of the spatial ellipse. A vertex in the + direction of (Y-axis) is set. On the other hand, the feature point determination condition of the fifth metatarsal bone of the left foot is set to a vertex in the - direction of the left/right axis.

Note that the method by which the condition input unit 23 acquires the feature point determination conditions is not limited to the above-described example. You may make it acquire point determination conditions. Also, in the above description, an example is shown in which the user determines the feature point determination conditions. It may be discriminated and a feature point determination condition may be determined.

The plane calculation unit 24 calculates (generates) a plane (plane A) that is a plane in a three-dimensional space that passes through the first body feature point and is perpendicular to a straight line that connects the first body feature point and the second body feature point. do. In this embodiment, the plane calculator 24 first calculates the three-dimensional coordinate value P ₁ ((P ₁ =(p _1x , p _1y , p _1z )) of the first body feature point and the three-dimensional coordinate value of the second body feature point. _A unit direction vector V ₍ V= _{( v x , v y} , v Then, the plane calculator 24 calculates a plane A passing through the three-dimensional coordinate _{value P1} of the first body feature point and perpendicular to the unit direction vector V (the normal vector plane) is calculated by the following equation (1).

FIG. 3 is a diagram showing an example of a leg portion according to the present embodiment. As shown in FIG. 3, in this embodiment, by using the formula (1), from the toe (first body feature point) to the ankle joint (second body feature point) to the toe (first body feature point) The equation for the plane A perpendicular to the vector V pointing to is calculated. It should be noted that the normal vector used when obtaining the plane A is a unit vector if it is a vector directed from the second body feature point to the first body feature point (or from the first body feature point to the second body feature point) is not limited to

The extraction unit 25 extracts, from among the plurality of body surface points (observation points) in the target segment, body surface points whose distance from the plane A calculated (generated) by the plane calculation unit 24 is within a predetermined range. It is extracted as a body surface point on A (hereinafter referred to as a “planar body surface point”). Specifically, the extraction unit 25 extracts the equation of the plane A (equation (1)) and the three-dimensional coordinate values ( _Xc , _Yc , _Zc) of each body surface point of the subject input by the position input unit 21. ) to calculate the distance between the plane A and each body surface point. Then, the extraction unit 25 regards body surface points (body surface points close to the plane A) for which the calculated distance is within a predetermined range (for example, several millimeters) as body surface points on the plane A, and Extract as points. Let the coordinate values of each point of the planar body surface points extracted in this way be (x _j , y _j , z _j ) (j is a natural number from 2 to N, N is the number of planar body surface points), A set of planar body surface points (x _j , y _j , z _j ) is hereinafter referred to as "point cloud A". Note that (x _j , y _j , z _j ) are three-dimensional coordinate values (X _c , Y _c , Z _c ).

The approximation unit 26 uses the three-dimensional coordinate values (x _j , y _j , z _j ) of the plurality of planar body surface points (point group A) extracted by the extraction unit 25 to obtain the point group A (point group A is located) is approximated to a spatial ellipse, which is an ellipse in three-dimensional space. By approximating the point group A to a spatial ellipse, the approximation unit 26 can acquire three-dimensional coordinate values of points on the spatial ellipse that approximates the point group A. FIG. In the present embodiment, the approximation unit 26 obtains an equation capable of acquiring three-dimensional coordinate values of points on a spatial ellipse that approximates the point group A. FIG.

Here, an ellipse in three-dimensional space (space ellipse) is rotated and moved with respect to an ellipse with no inclination (hereinafter referred to as “plane ellipse”) with the center as the origin in two-dimensional space (XY plane). It can be expressed as an ellipse obtained by performing translational movement. In other words, it is possible to move (coordinate transformation) a point on the planar ellipse to a point on the spatial ellipse by rotating and translating the point on the planar ellipse. Therefore, in the present embodiment, the space ellipse that approximates the point group A is assumed to be a plane ellipse moved in a three-dimensional space, and the approximation unit 26 obtains the three-dimensional coordinate values of the points on the space ellipse using an equation , an equation for a planar ellipse that matches the spatial ellipse by moving in three-dimensional space, and an equation for transforming a point on the planar ellipse to a point on the spatial ellipse (hereinafter referred to as a "transformation equation") are obtained.

In addition, in this embodiment, the equation shown in the following equation (2) is used as the equation of the planar ellipse. In equation (2), a and b are either the major radius or the minor radius of the ellipse, and S is the enlargement magnification (including 1) of the spatial ellipse based on the ellipse whose area is 1. , abπ=1 and z=0.

Further, when the point Q' on the plane ellipse is rotated by the rotation matrix M and translated by the translation vector T, and as a result, the point Q' is moved (transformed) to the point Q on the spatial ellipse, The three-dimensional coordinate values (x, y, z) of the point Q on the spatial ellipse are the coordinate values (x ^'' , y ^'' , z ^'' ) of the point Q', the rotation matrix M, and the translation vector T (T =(x ^''' , y ^''' , z ^''' )) and is calculated by the following equation (3). Equation (3) is an equation (transformation equation) for transforming (moving) points on the planar ellipse to points on the spatial ellipse.

In this embodiment, the rotation matrix M is a rotation matrix for rotating α around an arbitrary axis (unit vector n=(n _x , _ny , _nz )) and is expressed by the following equation (4).

In the present embodiment, in order to obtain the above-described planar ellipse equation (equation (2)) and transformation equation (equation (3)) that match the space ellipse by moving in the three-dimensional space, rotation by the rotation matrix M A point group A′ (point group before rotational movement and translational movement) that moves to point group A by performing movement and translational movement by translation vector T is optimized so as to be approximated by a plane ellipse. That is, optimization is performed to minimize the error between the point group A' and the plane ellipse equation. The approximation unit 26 first inputs the coordinate values (x _j , y _j , z _j ) of each point of the point group A into the following equation (5), thereby obtaining the coordinate values of each point of the point group A as , to the coordinate values (x _j ^″ , y _j ^″ , z _j ^″ ) of each point in the point group A′. In equation (5), M is a rotation matrix representing the amount of rotation for point group A' to move to point group A, and (x ^''' , y ^''' , z ^''' ) is It is a translation vector representing the amount of translational movement (parallel movement) for point group A′ to move to point group A. FIG.

Then, the approximation unit 26 calculates the coordinate values (x _j ^″ , y _j ^″ , z _j ^″ ) of the point group A′ represented by Equation (5) and the plane ellipse equation represented by Equation (2) as By optimizing to minimize the error between the point group A' and the plane ellipse, the equation of the plane ellipse and the transformation equation are obtained. In this embodiment, as the error (error function) between the point group A' and the plane ellipse, the sum of the absolute values of the errors between each point of the point group A' and the plane ellipse shown in the following equation (6) Use The approximation unit 26 optimizes each parameter by optimizing so as to minimize the error function shown in Equation (6) (determination of optimum parameters). The parameters optimized to minimize the error function shown in Equation (6) are the translation vector components (x ^''' , y ^''' , z ^''' ), the unit vector n are the components (n _x , n _y , n _z ), α, a, b, S of .

In this way, the approximation unit 26 optimizes each parameter so that the plane ellipse best approximates the point group A' by optimizing to minimize the error function shown in Equation (6). It is possible to calculate (determine) the values of (x ^''' , y ^''' , z ^''' , _nx , _ny , _nz , α, a, b, S). From this, it becomes possible to calculate the plane ellipse equation (formula (2)) and the transformation equation (formula (3)), and to approximate the point group A to a spatial ellipse. Any optimization method may be used as the method of minimizing the error function shown in Equation (6). For example, a method using a solver, which is a function of spreadsheet software, may be used. good. Further, the error function to be minimized is not limited to the error function shown in Equation (6). For example, it may be the sum of squares of the error between the point group A' and the plane ellipse (sum of squares error).

Further, in this embodiment, by obtaining the equation of the plane ellipse and the transformation equation, it is possible to acquire the three-dimensional coordinate values of the points on the spatial ellipse that approximate the point group A. Any method, such as directly determining the equation of the spatial ellipse, may be used as long as the value can be obtained. Also, under the constraint that a spatial ellipse exists on the plane A, the point group A may be approximated to a spatial ellipse.

The position estimation unit 27 estimates the position (three-dimensional coordinate value) of the third body feature point based on the space ellipse that approximates the point group A and the feature point determination condition (calculation of the estimated coordinate value). In this embodiment, the position estimating unit 27 inputs the coordinate values on the plane ellipse corresponding to the feature point determination conditions to the transformation equation (equation (3)) for transforming points on the plane ellipse into points on the spatial ellipse. By doing so, the coordinate values on the space ellipse are calculated, and the calculated coordinate values on the space ellipse are determined as the estimated coordinate values (three-dimensional coordinate values) of the third body feature point. Specifically, the position estimating section 27 calculates (determines) the coordinate values (p, q, r) on the plane ellipse of the third body feature point corresponding to the feature point determination condition. Then, the position estimating unit 27 inputs the calculated (p, q, r) to the coordinate values (x ^'' , y ^'' , z ^'' ) on the plane ellipse in the equation (3) so that the spatial ellipse The upper coordinate values (x, y, z) are calculated, and the calculated coordinate values (x, y, z) on the space ellipse are determined as the three-dimensional coordinate values (estimated coordinate values) of the third body feature point. .

As described above, when the fifth metatarsal bone in the foot is the third body feature point, as a feature point determination condition, the "vertex of the major axis (outer side of the body)" of the spatial ellipse that approximates the body surface point of the foot direction vertex)” is set (see FIG. 3). Therefore, for example, if the x-axis in the two-dimensional space is the major axis direction of the plane ellipse (a>b) and the + direction of the x-axis is the right horizontal direction as seen from the range image sensor, the position estimation unit 27 detects the right foot The coordinate values (p, q, r) on the plane ellipse of the fifth metatarsal bone are calculated (determined) by (p, q, r) = (-aS, 0, 0), and the fifth middle of the left foot The coordinate values (p, q, r) on the planar ellipse of the foot bone are calculated by (p, q, r)=(aS, 0, 0). Here, S is an enlargement magnification based on an ellipse having an area of 1, as described above. Then, the position estimating unit 27 assigns coordinates (x ^'' , y ^'' , z ^'' ) on the planar ellipse in Equation (3) to By inputting the coordinate values (aS, 0, 0) and (-aS, 0, 0) respectively, the three-dimensional coordinate values (x, y, z) of the fifth metatarsal of the right foot and left foot are calculated. .

Note that, for example, the feature point determination condition is a condition indicating that the third body feature point is the vertex of the short axis of the space ellipse, and the x-axis in the two-dimensional space is the major axis direction of the planar ellipse (a>b). , the coordinate values (p, q, r) on the plane ellipse of the third body feature point are (p, q, r) = (0, bS, 0) or (0, -bS, 0) Calculated (determined) by Further, when the spatial ellipse equation is directly calculated by the approximation unit 26, the three-dimensional coordinate values of the third body feature points may be calculated (estimated) using the spatial ellipse equation and feature point determination conditions. As described above, in this embodiment, the position of the desired third body feature point is determined on the plane ellipse by obtaining the equation of the plane ellipse and the transformation equation as equations capable of acquiring the three-dimensional coordinate values of the points on the space ellipse. can be specified as coordinate values (for example, (aS, 0, 0)).

The coordinate system generation unit 28 generates a coordinate system (orthogonal coordinate system) of the body segment by determining the three axes of the body segment. The coordinate system generation unit 28 uses the three-dimensional coordinate values of the first body feature point and the second body feature point and the estimated three-dimensional coordinate value of the third body feature point to create the coordinate system of the body segment of interest. Generate. For example, the coordinate system generation unit 28 sets the unit vector from the second body feature point to the first body feature point as the first axis, the first body feature point, the second body feature point, and the estimated third body feature point as Determine the unit normal vector of the plane passing through as the second axis, and the unit vector perpendicular to the first and second axes as the third axis. For example, if the third body feature point is the fifth metatarsal bone, the unit vector from the ankle joint to the toe is the unit normal of the plane passing through the first axis, the ankle joint, the toe and the estimated fifth metatarsal point. The vector is determined as the second axis and the unit vector perpendicular to the first and second axes as the third axis.

From this, the coordinate system generation unit 28 determines (generates) the coordinate system composed of the determined first, second, and third axes as the coordinate system of the body segment of interest. In addition, the three axes in the coordinate system of the target body segment are not limited to the three axes shown above, and any three axes based on the first body feature point, the second body feature point, and the third body feature point It's okay. In addition to the target body segment, the coordinate system generation unit 28 may also generate a coordinate system for body segments other than the target body segment.

The angle calculation unit 29 calculates the Euler angle between the coordinate system of the target body segment generated by the coordinate system generation unit 28 and another coordinate system (orthogonal coordinate system) to calculate the motion of the target body segment. Calculate the joint angle associated with Here, the movement of the target segment includes varus, valgus, adduction, abduction, flexion, extension, internal rotation, external rotation, pronation, supination, plantarflexion, dorsiflexion, etc. varus angle, valgus angle, adduction angle, abduction angle, flexion angle, extension angle, internal rotation angle, external rotation angle, pronation angle, supination angle, Angles such as plantar flexion angle and dorsiflexion angle. Calculate Euler angles between another coordinate system (reference coordinate system) suitable for calculating a desired (predetermined) joint angle and the coordinate system of the body segment of interest generated by the coordinate system generator 28 This makes it possible to calculate the desired joint angle. Note that the other coordinate system may be any coordinate system such as a coordinate system of other body segments, such as a body segment adjacent to the target body segment, or a global coordinate system. Further, as the other coordinate system, another coordinate system stored in the storage device 14 may be used, or the coordinates of the body segment other than the target body segment generated by the coordinate system generation unit 28 may be used. system may be used.

The following is a case of calculating the varus/valgus angle of the foot (ankle joint) using the estimated three-dimensional coordinate value of the fifth metatarsal when the third body feature point is the fifth metatarsal. Illustrate. In this embodiment, the angle of varus/valgus of the ankle joint means the angle of rotation about the axis from the heel to the second metatarsal bone.

FIG. 4 is a diagram showing an example of joint angle calculation according to the present embodiment. FIG. 4 shows the coordinate system (X-axis, Y-axis, Z-axis) in the body segment (foot) of interest, and three other coordinate systems (reference coordinate system): the hip joint, the knee joint, and the ankle joint. A coordinate system consisting of three axes (X'-axis, Y'-axis, Z'-axis) determined by points is shown. As shown in FIG. 4, in the present embodiment, the coordinate system in the body segment of interest is defined as a horizontal plane passing through the first body feature point, the second body feature point, and the third body feature point. A coordinate system with the front-rear axis (X-axis) extending from the point to the first body feature point. Further, in the present embodiment, the reference coordinate system is a virtual segment (rigid body) coordinate system in which the lower limbs other than the foot are treated as one body segment (rigid body). The sagittal plane is the plane passing through the joint and the ankle joint, and the vertical axis (Z′-axis) is the direction from the ankle joint to the knee joint. Since the knee joint is a uniaxial joint, it is possible to treat the plane formed by the hip joint, knee joint, and ankle joint as one virtual rigid body.

The angle calculator 29 calculates Euler angles (rotational angles) between the coordinate system of the body segment (leg) of interest and the reference coordinate system shown in FIG. Specifically, the angle calculator 29 rotates the coordinate system of the body segment of interest around its own coordinate axes (X axis (front-back axis), Y axis (left-right axis), Z-axis (vertical axis)). Calculate the Euler angles for matching with the coordinate system of the reference segment. In this embodiment, the angle calculator 29 is a rotation matrix for matching the coordinate system of the target body segment with the coordinate system of the reference body segment. By calculating a rotation matrix Q about three axes, where the angle is θ and the rotation angle about the Z axis is Ψ, the Euler angles for matching with the coordinate system of the reference body segment are calculated. From this, the angle calculation unit 29 can determine (calculate) the calculated rotation angle φ about the X-axis (front-rear axis) as the varus-valgus angle of the ankle joint. Although the above rotation order is used for the Euler angles this time, the rotation order is not limited to this.

In this embodiment, the reference coordinate system for calculating the Euler angle with respect to the coordinate system of the target body segment for calculating the varus/valgus angle is determined by the three points of the hip joint, the knee joint, and the ankle joint. Although the coordinate system is used, the method of obtaining the reference coordinate system is not limited to this method. For example, using the above-described method of estimating the position of a body feature point, the first side position of another body segment (hereinafter referred to as a "reference body segment") different from the target body segment is identified. Based on the one body feature point and the second body feature point specifying the position of the second side, the coordinate position of the third body feature point on the reference body segment is estimated. Then, the coordinate system determined by the three points of the first body feature point, the second body feature point, and the third body feature point for the reference body segment may be used as the reference coordinate system.

For example, the first body feature point is the knee joint, the second body feature point is the ankle joint, and the feature point determination condition is the vertex of the short axis (vertex in the lateral direction of the body). Three-dimensional coordinate values of the lateral epicondyle of the joint are calculated. In addition to the coordinate system of the target segment, the coordinate system generation unit 28 creates a reference coordinate system (of the crus, which is the reference segment) by three points of the knee joint, the ankle joint, and the lateral epicondyle of the knee joint. coordinate system). Specifically, a coordinate system is generated in which a plane passing through the knee joint, the ankle joint, and the lateral epicondyle of the knee joint is the frontal plane, and the vertical axis is the direction from the ankle joint to the knee joint. Then, the angle calculator 29 calculates the Euler angle between the coordinate system of the target body segment generated by the coordinate system generator 28 and the coordinate system of the reference body segment, thereby calculating the varus/valgus of the foot. It becomes possible to calculate the angle.

The angle calculation unit 29 may calculate the calculated Euler angle itself as the desired joint angle, or may invert the polarity (plus or minus sign) of the calculated Euler angle and convert the inverted angle to the desired joint angle. may be calculated as a joint angle of For example, the varus/valgus angle of the ankle joint calculated by the method described above is calculated (output) with the anteroposterior axis of the foot being + in the direction from the ankle joint to the toe, and the rotation in the right screw direction being +. Here, in the case of the right foot, the direction of rotation of the foot (ankle joint) around the front-rear axis is the negative direction when the foot is valgus, and the direction of rotation around the front-rear axis of the foot is the + direction when the foot is inversion. . On the other hand, in the case of the left foot, the direction of rotation of the foot (ankle joint) around the front-rear axis is + in the case of valgus, and the direction of rotation around the front-rear axis of the foot is the - direction in the case of inversion. Therefore, when it is desired to calculate (output) a valgus angle with the valgus direction being positive, the angle calculator 29 multiplies the calculated Euler angle about the front-rear axis of the right foot by -1 to obtain the valgus angle of the right foot. is calculated, and the calculated Euler angle about the front-rear axis of the left foot is multiplied by +1 to calculate the valgus angle of the left foot. In this way, the angle calculation unit 29 may perform various processes to calculate the desired joint angles by matching the polarities of the desired joint angles with the polarities of the Euler angles between the coordinate systems. .

As described above, this embodiment exemplifies a method of estimating the position of the fifth metatarsal (three-dimensional coordinate values) using the three-dimensional coordinate values of the ankle joint and the toe acquired by the three-dimensional measuring device 3. bottom. However, the target to be estimated (third body feature point) is not limited to the fifth metatarsal bone, and may be any other body feature point. For example, in the present embodiment, the position of the first metatarsal bone is determined by setting the feature point determination condition to the vertex of the long axis (vertex in the lateral direction of the body) instead of the vertex of the long axis (vertex in the lateral direction of the body). can be estimated. Therefore, for example, the coordinate values (p, q, r) on the plane ellipse corresponding to the feature point determination condition of the fifth metatarsal bone of the right foot are (p, q, r) = (-aS, 0, 0) In this case, the coordinate values (p, q, r) on the plane ellipse corresponding to the feature point determination condition of the first metatarsal bone of the right foot are (p, q, r)=(aS, 0, 0). In this way, by setting the target segment as the foot, the first body feature point as the toe, and the second body feature point as the ankle joint, the position (three-dimensional coordinate value) can be estimated.

Also, by setting the ankle joint as the first body feature point and the knee joint as the second body feature point, it is possible to estimate the positions of the medial malleolus and the lateral malleolus. Further, by setting the wrist joint as the first body feature point and the elbow joint as the second body feature point, it is possible to estimate the positions of the radial styloid process and the ulnar styloid process. Further, by setting the knee joint as the first body feature point and the hip joint as the second body feature point, it is possible to estimate the positions of the femoral medial epicondyle and femoral lateral epicondyle. Furthermore, by setting the elbow joint as the first body feature point and the shoulder joint as the second body feature point, it is possible to estimate the positions of the medial epicondyle of the humerus and the lateral epicondyle of the humerus. In each case, the feature point determination condition can be appropriately set by a person skilled in the art.

<Process flow>
Next, a flow of feature point position estimation processing executed by the feature point position estimation device 1 according to the present embodiment will be described. Note that the specific content and processing order of the processing described below are examples for carrying out the present disclosure. Specific processing contents and processing order may be appropriately selected according to the embodiment of the present disclosure.

FIG. 5 is a flowchart showing an overview of the flow of feature point position estimation processing according to this embodiment. The processing shown in this flowchart is triggered when an instruction to acquire coordinate data of an object (subject), an instruction to estimate a body feature point, or the like is received from a user or the like in the feature point position estimation device 1. is executed as In the following, the third body feature point is assumed to be the fifth metatarsal bone in the foot, and the process for estimating the position of the fifth metatarsal bone will be exemplified.

In step S101, the three-dimensional coordinate values of predetermined body feature points (first body feature point, second body feature point) are input to the three-dimensional measuring device 3 . The position input unit 21 specifies the toe (first body feature point) that specifies the position of the first side of the foot (target segment) where the fifth metatarsal of the subject is located, and the position of the second side. The three-dimensional coordinate values of the ankle joint (second body feature point) are obtained from the three-dimensional measuring device 3 and input. After that, the process proceeds to step S102.

In step S102, the three-dimensional coordinate values of the body surface points (observation points) of the object (subject) are input. The position input unit 21 acquires two-dimensional coordinate values of a plurality of body surface points on the subject's foot (target segment) from the three-dimensional measuring device 3, and uses the conversion function stored in the storage unit 22. By converting the two-dimensional coordinate values of the body surface points into three-dimensional coordinate values, the three-dimensional coordinate values of a plurality of body surface points on the feet of the subject are acquired (input). After that, the process proceeds to step S103.

In step S103, a feature point determination condition is input. The condition input unit 23 inputs the fifth metatarsal bone (third body feature point) to indicate that the fifth metatarsal bone (third body feature point) is positioned at the vertex of the major axis of the spatial ellipse (vertex in the lateral direction of the body) that approximates the body surface point of the foot. Enter the criteria for determining the feature points of the five metatarsals. After that, the process proceeds to step S104. The order of steps S101 to S103 is random.

In step S104, a plane (plane A) passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is calculated. The plane calculation unit 24 uses the three-dimensional coordinate values of the toe (first body feature point) and the ankle joint (second body feature point) input in step S101 to obtain a plane passing through the toe and from the ankle joint. Compute the equation for the plane A perpendicular to the vector V pointing to the toes. After that, the process proceeds to step S105.

In step S105, body surface points (point group A) whose distance from plane A is within a predetermined range are extracted. The extraction unit 25 uses the equation of the plane A (equation (1)) calculated in step S104 and the three-dimensional coordinate values of the points on the body surface input in step S102 to obtain a predetermined distance from the plane A. The body surface points within the range are extracted as body surface points on the plane A (point group A). After that, the process proceeds to step S106.

In step S106, the point group A is approximated to a spatial ellipse. The approximation unit 26 approximates the point group A with a spatial ellipse, and uses an equation capable of acquiring the three-dimensional coordinate values of the points on the spatial ellipse that approximates the point group A by moving the spatial ellipse on the three-dimensional space. Find the equation of the planar ellipse consistent with , and the transformation equation that transforms points on the planar ellipse to points on the spatial ellipse. After that, the process proceeds to step S107.

In step S107, the position of the third body feature point is estimated based on the feature point determination condition and the spatial ellipse. The position estimating unit 27 obtains the coordinate values on the plane ellipse corresponding to the fifth metatarsal feature point determination condition input in step S103, and inputs the obtained coordinate values into the conversion equation obtained in step S106. By doing so, the three-dimensional coordinate value of the fifth metatarsal is calculated. Then, the position estimator 27 determines the calculated three-dimensional coordinate value as the estimated position of the fifth metatarsal bone. After that, the processing shown in this flowchart ends.

FIG. 6 is a flowchart showing an overview of the flow of angle calculation processing according to this embodiment. The process shown in this flowchart is executed when the feature point position estimation device 1 estimates the position of the third body feature point in the body segment of interest. In the following, the fifth metatarsal bone in the foot is assumed to be the third body feature point, and the processing for calculating the varus/valgus angle of the ankle joint will be exemplified.

In step S201, a Cartesian coordinate system of the body segment of interest is generated. The coordinate system generation unit 27 sets the unit vector from the ankle joint to the toe as the first axis, and the unit normal vector of the plane passing through the ankle joint, the toe, and the estimated fifth metatarsal bone point as the second axis and the first axis. and the unit vector perpendicular to the second axis is determined as the third axis, and a coordinate system consisting of the first, second and third axes is generated as the coordinate system of the body segment of interest. After that, the process proceeds to step S202.

In step S202, joint angles are calculated. The angle calculation unit 29 calculates the Euler angles between the coordinate system of the target body segment (leg) generated in step S201 and the reference coordinate system (see FIG. 4). A rotation angle φ about the X-axis (anterior-posterior axis) is determined (calculated) as the varus-valgus angle of the ankle joint. After that, the processing shown in this flowchart ends.

According to the system shown in this embodiment, the three-dimensional By using the coordinate values, the three-dimensional coordinate values of a plurality of body surface points in the body segment of the target, and the feature point determination condition, the three-dimensional coordinate position of the third body feature point located in the body segment of the target is determined. It is possible to estimate

As described above, the distance image sensor (three-dimensional measuring device) can acquire the position coordinates of predetermined body feature points with the distance image sensor. For example, in the case of lower limbs, the range image sensor normally outputs three-dimensional coordinate values of knee joints, ankle joints, and toes, which are body feature points set in the range image sensor. Although it is possible to calculate plantar flexion/dorsiflexion angles and adduction/abduction angles of the ankle joint using projection angles from the three-dimensional coordinate values of these body feature points, It is difficult to correctly calculate the varus/valgus angle. In order to calculate the varus/valgus angle of the ankle joint, as described above, it is necessary to obtain the Euler angles between the body segments (between the coordinate systems of the body segments). ), it is necessary to acquire the three-dimensional coordinate values of another point (body feature point). In this way, in order to calculate the joint angles about the three axes, in the body segment where only the coordinate values of two body feature points are output (for example, the foot where only the coordinate values of the ankle joint and the toe are output), another It is necessary to estimate the positions of body feature points (virtual points).

According to the system shown in this embodiment, as described above, three of the first body minutia to specify the position of the first side of the target segment and the second body minutia to specify the position of the second side. From the dimensional coordinate values, it is possible to estimate the 3D coordinate position of the third body feature point located on the body segment of the subject. Therefore, it is possible to estimate the position of another body feature point (body feature point not predetermined by the range image sensor) in a body segment where only the three-dimensional coordinate values of two body feature points are output from the range image sensor. . As a result, it is possible to generate a coordinate system for the relevant body segment, and by calculating the Euler angles between the coordinate system of the relevant body segment and another coordinate system, joint angles that are difficult to calculate using projection angles can be calculated. It becomes possible to In this way, by making it possible to calculate the joint angle, which is usually difficult to calculate, it becomes possible to deepen understanding of the living body, and it becomes possible to contribute to rehabilitation medicine, orthopedic treatment, and the like.

1 feature point position estimation device 3 three-dimensional measuring device 9 system

Claims (14)

A feature point position estimation device for estimating a three-dimensional coordinate value of a predetermined body feature point located in a target segment of an object,
three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment of the object and a second body feature point specifying the position of the second side of the body segment of the object; position input means for inputting three-dimensional coordinate values of body surface points;
A plane in a three-dimensional space passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is defined as the first body feature point and the second body feature point. Plane calculation means for calculating using three-dimensional coordinate values of body feature points;
extracting means for extracting a plurality of planar body surface points whose distance from the plane is within a predetermined range from the plurality of body surface points;
approximation means for approximating the plurality of planar body surface points to a spatial ellipse, which is an ellipse in a three-dimensional space, using the three-dimensional coordinate values of the plurality of planar body surface points;
condition input means for inputting a condition indicating at which position on the spatial ellipse the predetermined body feature point exists;
position estimating means for estimating three-dimensional coordinate values of the predetermined body feature points based on the spatial ellipse and the conditions;
A feature point position estimation device comprising: The approximation means obtains an equation capable of obtaining three-dimensional coordinate values of points on the spatial ellipse,
The feature point position estimation device according to claim 1. The condition was assumed to approximate the shape of the body surface of the target segment on a plane that passes through the first body feature point and is perpendicular to a straight line that connects the first body feature point and the second body feature point with an ellipse. In the case, a condition determined based on where the third body feature point is located in the ellipse,
The feature point position estimation device according to claim 1 or 2. The condition is a condition indicating that the predetermined body feature point is one of the vertices of the spatial ellipse.
The feature point position estimation device according to any one of claims 1 to 3. The three-dimensional coordinate values of the first body feature point and the second body feature point are coordinate values obtained by a three-dimensional measuring device that measures the three-dimensional coordinate values of predetermined body feature points of an object. ,
The feature point position estimation device according to any one of claims 1 to 4. Using the three-dimensional coordinate values of the first body feature point and the second body feature point and the estimated three-dimensional coordinate value of the predetermined body feature point, an orthogonal coordinate system of the target body segment is generated. further comprising a coordinate system generating means;
The feature point position estimation device according to any one of claims 1 to 5. Further comprising angle calculation means for calculating a joint angle associated with the movement of the target body segment by calculating Euler angles between the Cartesian coordinate system of the target body segment and another Cartesian coordinate system,
The feature point position estimation device according to claim 6. the body segment of the subject is one of a plurality of body segments of the subject;
Each of the position input means, the plane calculation means, the approximation means, the condition input means, the feature point position estimation means, and the coordinate system generation means, in addition to the target body segment, performing processing on other body segments different from the body segment of the subject;
The angle calculation means calculates a joint angle associated with the movement of the target body segment by calculating an Euler angle between the orthogonal coordinate system of the target body segment and the orthogonal coordinate system of the other body segment.
The feature point position estimation device according to claim 7. The approximation means includes, as equations capable of obtaining three-dimensional coordinate values of points on the space ellipse, an equation of a two-dimensional space ellipse that matches the space ellipse by moving in a three-dimensional space, and the two-dimensional Finding a transformation equation that transforms a point on a space ellipse to a point on said space ellipse;
The feature point position estimation device according to claim 2. The position estimating means estimates the three-dimensional coordinate value of the predetermined body feature point by inputting the coordinate value on the ellipse of the two-dimensional space corresponding to the input condition into the transformation equation.
The feature point position estimation device according to claim 9 . the segment is a foot,
the first body feature point is a toe;
the second body feature point is an ankle joint;
The predetermined body feature point is an edge in the width direction on the toe side of the foot,
The feature point position estimation device according to any one of claims 1 to 10. the predetermined body feature point is the fifth metatarsal;
The condition is a condition indicating that the fifth metatarsal bone, which is the predetermined physical feature point, is the vertex in the lateral direction of the major axis of the spatial ellipse;
The position estimation means estimates a three-dimensional coordinate value of the fifth metatarsal bone, which is the predetermined physical feature point.
The feature point position estimation device according to claim 11. A computer for estimating three-dimensional coordinate values of predetermined body feature points located in target body segments of an object,
three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment of the object and a second body feature point specifying the position of the second side of the body segment of the object; a position input step of inputting three-dimensional coordinate values of body surface points;
A plane in a three-dimensional space passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is defined as the first body feature point and the second body feature point. a plane calculation step of calculating using the three-dimensional coordinate values of the body feature points;
an extracting step of extracting a plurality of planar body surface points whose distance from the plane is within a predetermined range from the plurality of body surface points;
an approximation step of approximating the plurality of planar body surface points to a spatial ellipse, which is an ellipse in a three-dimensional space, using the three-dimensional coordinate values of the plurality of planar body surface points;
a condition input step of inputting a condition indicating at which position on the spatial ellipse the predetermined body feature point exists;
a position estimation step of estimating a three-dimensional coordinate value of the predetermined body feature point based on the spatial ellipse and the condition;
A feature point location estimation method that performs a computer for estimating three-dimensional coordinate values of predetermined body feature points located on target body segments of an object;
three-dimensional coordinate values of a first body feature point specifying the position of the first side of the body segment of the object and a second body feature point specifying the position of the second side of the body segment of the object; position input means for inputting three-dimensional coordinate values of body surface points;
A plane in a three-dimensional space passing through the first body feature point and perpendicular to a straight line connecting the first body feature point and the second body feature point is defined as the first body feature point and the second body feature point. Plane calculation means for calculating using three-dimensional coordinate values of body feature points;
extracting means for extracting a plurality of planar body surface points whose distance from the plane is within a predetermined range from the plurality of body surface points;
approximation means for approximating the plurality of planar body surface points to a spatial ellipse, which is an ellipse in a three-dimensional space, using the three-dimensional coordinate values of the plurality of planar body surface points;
condition input means for inputting a condition indicating at which position on the spatial ellipse the predetermined body feature point exists;
position estimating means for estimating three-dimensional coordinate values of the predetermined body feature points based on the spatial ellipse and the conditions;
A program to function as

Images