JP7089238B2 - Center of gravity position estimation device, center of gravity position estimation method, program - Google Patents

Description

The present invention relates to a technique for estimating the position of the center of gravity of the human body from a multi-viewpoint image.

The position of the three-dimensional center of gravity (COM) of the human body is important for analyzing human movements in various fields such as rehabilitation and sports. Generally, in order to estimate the position of the three-dimensional center of gravity, a two-dimensional position (COP: Center of Projection) obtained by projecting the position of the three-dimensional center of gravity obtained from the force plate in the vertical direction is often used. In order to use this force plate, the person to be analyzed must be on the plate. Therefore, while the two-dimensional position can be estimated without any problem in a laboratory environment or a controllable environment, the two-dimensional position cannot be estimated in an environment where it is difficult to install a plate as in an actual game.

In response to this problem, in Non-Patent Document 1, by using images taken from a plurality of viewpoints (hereinafter referred to as multi-viewpoint images), a force plate can be used, or some kind of sensor can be attached to a person to be analyzed. We are proposing a method to estimate the position of the three-dimensional center of gravity of the human body non-invasively without doing so. In this method, the three-dimensional region representing the area occupied by the human body in space is restored as a collection of multiple unit cubes (voxels) using the visual volume crossing method, and each of the human bodies is relative to the voxels contained in the three-dimensional region. Appropriate weights are given according to the part, and the weighted average calculated using the voxel position coordinates and weights is estimated as the three-dimensional center of gravity position.

Tomoya Toriuchi, Shohei Mori, Hideo Saito, Kosuke Takahashi, Bullet Mikami, Mariko Ichikawa, Hideaki Kizen, "Study of Method for Estimating Center of Gravity of Human Body Using Multiple Viewpoint Images", Shingaku Giho, Vol.117, No.252, MVE2017-28, pp.19-23, October 2017.

As described above, the visual volume crossing method is used in the method of Non-Patent Document 1. Therefore, as shown in the bird's-eye view of FIG. 1 (a), when there is no blind spot from the camera, the three-dimensional region of the human body can be appropriately restored, but it is shown in the bird's-eye view of FIG. 1 (b). If there is a blind spot from the camera, the three-dimensional area of the human body cannot be restored correctly. Therefore, if there is a region that cannot be observed from the camera, there is a problem that the estimation accuracy of the three-dimensional center of gravity position is lowered due to the inability to correctly restore the three-dimensional region of the human body. ..

Therefore, an object of the present invention is to provide a technique for accurately estimating the position of the three-dimensional center of gravity of the human body from a multi-viewpoint image even when there is a region that cannot be observed by the camera.

One aspect of the present invention constitutes a three-dimensional region representing an region occupied by the body of the person at time t from a multi-viewpoint image taken by a plurality of cameras installed so as to surround the person as a subject. From the multi-viewpoint video, the three-dimensional model generator that generates a set of boxel data including at least the boxel position coordinates, which are the three-dimensional coordinates of the boxel position, as a three-dimensional model at time t, and the joint of the person at time t. A box cell that constitutes the three-dimensional region from a three-dimensional joint position estimation unit that estimates three-dimensional coordinates representing positions as joint position coordinates at time t, a three-dimensional model at time t, and joint position coordinates at time t. By determining a part identifier for identifying a part of the human body including the boxel and a weight for the boxel, a set of weighted boxel data including at least the boxel position coordinates, the part identifier, and the weight is obtained. The weighted 3D model generator generated as a weighted 3D model at time t, the part 3D center of gravity position is the weighted average of the part of the person's body, and the 3D center of gravity position is the weighted average of the person's body. Then, the weighted three-dimensional model at the time t and the part three-dimensional center of gravity position of the part identifier i (i = 1, ..., N _p , N _p are the number of parts of the human body) at the time before the time t are determined. Includes a 3D center of gravity position estimation unit that estimates the position of the 3D center of gravity at time t from the weighted 3D model (hereinafter referred to as the reference weighted 3D model) at the time of.

According to the present invention, it is possible to accurately estimate the position of the three-dimensional center of gravity of the human body from a multi-viewpoint image even when there is a region that cannot be observed by the camera.

The figure which shows the relationship between the multiple cameras and the area of the restored human body. The block diagram which shows the structure of the center of gravity position estimation apparatus 100. The flowchart which shows the operation of the center of gravity position estimation apparatus 100. The figure which shows the state of a three-dimensional area and a voxel which is a component thereof. The figure which shows how the size of a voxel is determined. The figure which shows an example of the part of a human body in a three-dimensional area. The block diagram which shows the structure of the 3D center of gravity position estimation part 140. The flowchart which shows the operation of the 3D center of gravity position estimation part 140.

Hereinafter, embodiments of the present invention will be described in detail. The components having the same function are given the same number, and duplicate explanations are omitted.

<First Embodiment>
In order to solve the above problems, in the embodiment of the present invention, the finding that "the position of the three-dimensional center of gravity of each part of the human body changes smoothly" (hereinafter referred to as "finding 1") and "the total weight of each part of the human body are The position of the three-dimensional center of gravity is estimated by incorporating the knowledge that "it is invariant in the time direction" (hereinafter referred to as "knowledge 2").

Hereinafter, the center of gravity position estimation device 100 will be described with reference to FIGS. 2 to 3. FIG. 2 is a block diagram showing the configuration of the center of gravity position estimation device 100. FIG. 3 is a flowchart showing the operation of the center of gravity position estimation device 100.

A multi-viewpoint image taken by a plurality of (that is, two or more) cameras is input to the center of gravity position estimation device 100. These cameras are set to shoot the same subject. Specifically, as shown in FIG. 1, a plurality of cameras may be installed so as to surround the person to be the subject. The image taken by the camera may be directly input to the center of gravity position estimation device 100, or may be once recorded on another external recording medium and then input to the center of gravity position estimation device 100. It is preferable that these cameras have been synchronized and geometrically calibrated. In this case, the images taken by the plurality of cameras are synchronized and geometrically calibrated multi-viewpoint images. Hereinafter, the number of cameras is N _c (N _c is an integer of 2 or more). Further, the frame number t (t = 1, 2, ..., That is, t is an integer of 1 or more), which is a number for identifying a frame that is an image constituting the video, is referred to as a time t.

As shown in FIG. 2, the center of gravity position estimation device 100 includes a three-dimensional model generation unit 110, a three-dimensional joint position estimation unit 120, a weighted three-dimensional model generation unit 130, a three-dimensional center of gravity position estimation unit 140, and recording. Includes part 190. The recording unit 190 is a component unit that appropriately records information necessary for processing. For example, the recording unit 190 records a multi-viewpoint video composed of N _c videos.

Hereinafter, the operation of the center of gravity position estimation device 100 will be described with reference to FIG. The three-dimensional model generation unit 110 takes the multi-viewpoint image recorded in the recording unit 190 as an input, and at time t, the third-order position of the voxel constituting the three-dimensional area representing the area occupied by the body of the person to be the subject in the space. A set of voxel data including at least the voxel position coordinates which are the original coordinates is generated and output as a three-dimensional model at time t (S110). The voxel data may include, for example, a voxel identifier for identifying a voxel, in addition to the voxel position coordinates. As shown in FIG. 4, the three-dimensional region is composed of voxels which are cubes having a predetermined length (hereinafter referred to as a unit length) as one side. The size (unit length) of the voxel depends on the resolution of the camera used for shooting, the area of the person in the image, the estimation accuracy of the expected three-dimensional center of gravity position, and the like. The size of the voxel is the size of the square in the area where the person exists when a 1-pixel square of the image taken by the camera C _k (k = 1, ..., N _c ) is back-projected into the real space. The length of the side is L _k , and it is desirable that it is min _k (L _k ) or less (see FIG. 5).

In addition, any method can be used to restore the three-dimensional region. For example, the visual volume crossing method used in Non-Patent Document 1, that is, a method based on Visual Hull can be used. The visual volume crossing method is a method of obtaining a three-dimensional region as an intersection of silhouettes of a subject reflected in a multi-viewpoint image, and is described in detail in Reference Non-Patent Document 1.
(Reference Non-Patent Document 1: Takashi Matsuyama, Xiaojun Wu, Takeshi Takai, and Shohei Nobuhara, “Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video”, Computer Vision and Image Understanding, Vol. 96, Issue 3, pp.393-434, Dec. 2004.)

The three-dimensional joint position estimation unit 120 takes the multi-viewpoint image input in S110 as input, estimates three-dimensional coordinates representing the position of the joint of the person who is the subject at time t as the joint position coordinates at time t, and outputs the coordinates ( S120). Any method can be used to estimate the joint position coordinates. For example, the same method as in Non-Patent Document 1 can be used. Specifically, first, the two-dimensional coordinates of each joint of the person concerned are estimated on the image at time t included in each video constituting the multi-viewpoint video. Next, the joint position coordinates, which are three-dimensional coordinates, are obtained by triangulation using the estimated two-dimensional coordinates. For the estimation of the two-dimensional coordinates of the joint, for example, the method shown in Reference Non-Patent Document 2 can be used.
(Reference Non-Patent Document 2: Zhe Cao, Tomas Simon, Shih-En Wei and Yaser Sheikh, “Realtime Multi-Person 2D Pose Optimization Using Part Affinity Fields”, CVPR 2017, 2017.)

The weighted three-dimensional model generation unit 130 inputs the three-dimensional model at the time t generated in S110 and the joint position coordinates at the time t estimated in S120, and includes the voxels for the voxels constituting the three-dimensional region. By determining the part identifier for identifying the part of the human body and the weight for the voxel, a set of weighted voxel data including at least the voxel position coordinates, the part identifier and the weight is generated as a weighted three-dimensional model at time t. , Output (S130). The weighted voxel data may include, for example, a voxel identifier in addition to the voxel position coordinates, the site identifier, and the weight.

Any method can be used as a method for determining the site identifier and the weight. For example, the same method as in Non-Patent Document 1 can be used. In Non-Patent Document 1, as shown in FIG. 6, the human body is divided into 10 parts (head, torso, upper left arm, left forearm, upper right arm, right forearm, left thigh, left calf, right thigh, right calf). Model. In general, the number of parts is not limited to 10, and can be N _p (N _p is an integer of 2 or more and represents the number of parts of the human body). Hereinafter, the site identifier for identifying each site is referred to as i (i = 1, ..., N _p ). In addition, the corresponding body part is given to the straight line connecting the part and the joint which is the node of the part. For example, a head is given to a straight line corresponding to the head (see FIG. 6). Then, for each voxel, a straight line that minimizes the distance between the straight line connecting the joints and the voxel is obtained, and the voxel is considered to be included in the body part corresponding to the straight line that minimizes this distance. Determine the site identifier of the site containing the voxel. Further, the weight for each voxel is determined based on the specific density of a general part of the human body as shown in Reference Non-Patent Document 3.
(Reference Non-Patent Document 3: PD Leva, “Adjustments to Zatsiorsky-Seluyanov's segment inertia parameters”, J. of Biomechanic, 29 (9), pp.1223-1230, 1996.)

The specific gravity of each part is recorded in the recording unit 190 in advance.

The three-dimensional center of gravity position estimation unit 140 has a weighted three-dimensional model at time t generated in S130 and a part identifier i (i = 1, ..., N _p ) at a time before the time t recorded in the recording unit 190. ) Part 3D center of gravity position and weighted 3D model at a predetermined time (hereinafter referred to as reference time) (hereinafter referred to as reference weighted 3D model) are input, and the 3D center of gravity position at time t is estimated. Output (S140). Here, the position of the three-dimensional center of gravity of the part is the weighted average of the part of the body of the person to be the subject, and the position of the three-dimensional center of gravity is the weighted average of the body of the person. Further, although the details will be described later, the reference weighted three-dimensional model is for showing the total weight of each part that does not change in the time direction in Finding 2.

Hereinafter, the three-dimensional center of gravity position estimation unit 140 will be described with reference to FIGS. 7 to 8. FIG. 7 is a block diagram showing the configuration of the three-dimensional center of gravity position estimation unit 140. FIG. 8 is a flowchart showing the operation of the three-dimensional center of gravity position estimation unit 140. As shown in FIG. 7, the three-dimensional center of gravity position estimation unit 140 includes a site three-dimensional center of gravity position estimation unit 141, a weighted three-dimensional model correction unit 142, and a three-dimensional center of gravity position calculation unit 143. The site three-dimensional center of gravity position estimation unit 141 and the weighted three-dimensional model correction unit 142 are constituent units that execute processing based on the knowledge 1 and the knowledge 2, respectively.

Hereinafter, the operation of the three-dimensional center of gravity position estimation unit 140 will be described with reference to FIG. The part three-dimensional center of gravity position estimation unit 141 inputs the part three-dimensional center of gravity position of the part identifier i (i = 1, ..., N _p ) at the time before the time t, and the part three-dimensional part of the part identifier i at the time t. The position of the center of gravity is estimated and output (S141). Further, the site three-dimensional center of gravity position estimation unit 141 records the site three-dimensional center of gravity position of the site identifier i at the estimated time t in the recording unit 190.

Any method can be used for estimation. For example, let N _t be an integer of 1 or more, and the part 3D center of gravity position C3D (i, tN _t ),…, C3D (i, t-) of N _t part identifiers i from time tN _t to time t-1. By applying a time-series filter such as a Kalman filter or a particle filter to 1), the part three-dimensional center of gravity position C3D (i, t) of the part identifier i at time t may be estimated. The appropriate value of N _t depends on the frame rate of the image, the speed of movement of the subject, the type of time-series filter, etc., but the position of the 3D center of gravity of the part at least 1 hour ago C3D (i, t-1) If there is), the position of the three-dimensional center of gravity of the site C3D (i, t) can be estimated.

Further, when t = 1, the position of the three-dimensional center of gravity of the part of the part identifier i at time t = 1 may be estimated by an appropriate method. For example, the site three-dimensional center of gravity position estimation unit 141 uses a weighted three-dimensional model at time t = 1 to weight a set of voxels contained in the site of the site identifier i using voxel position coordinates and weights. The attached average may be calculated, and the value may be set as the site three-dimensional center of gravity position C3D (i, 1) of the site identifier i.

The weighted three-dimensional model correction unit 142 has the part three-dimensional center of gravity position of the part identifier i (i = 1, ..., N _p ) at the time t estimated in S141, and the reference weighted third order recorded in the recording unit 190. The original model and the weighted 3D model at time t generated in S130 are input, and the modified weighted 3D model at time t is generated and output (S142). Here, the modified weighted 3D model at time t is generated by removing the weighted voxel data determined by a predetermined method from the weighted 3D model at time t, and is generated as a set at time t. It will be included in the weighted 3D model. Further, the weighted three-dimensional model correction unit 142 records the corrected weighted three-dimensional model at the generated time t in the recording unit 190 as the weighted three-dimensional model at the time t. The reason why the weighted three-dimensional model at time t is recorded is that, as described below, the weighted three-dimensional model at the time corresponding to the reference time is used as the reference weighted three-dimensional model.

Hereinafter, the generation method will be specifically described. Here, for i = 1, ..., N _p , the number of voxels contained in the site of the site identifier i at time t is N _{i, t} , and the voxels contained in the site of site identifier i at time t are v. (i, j, t) (j = 1,…, N _{i, t} ), the voxel position coordinates of the voxel are expressed as p (i, j, t), and the weight of the voxel is expressed as w _{i, j, t} . To. That is, p (i, j, t) is the position coordinate of the j-th voxel contained in the part of the part identifier i at time t, and w _{i, j, t} is the j-th contained in the part of the part identifier i at time t. Represents the weight given to the voxel of. In general, when a three-dimensional region is restored by using the visual volume crossing method, a region larger than the region actually occupied by the human body is restored as a three-dimensional region. Therefore, here, the weighted 3D model is modified by deleting voxels v (i, j, t) that are not included in the actual area occupied by the human body (that is, unnecessary).

Let N _i be the number of voxels contained in the site of the site identifier i in the reference weighted 3D model. The reference weighted three-dimensional model is a weighted three-dimensional model at the reference time t _s (t _s is a predetermined integer of 1 or more). The reference time t _s may be selected in any way. For example, the user may specify it separately. However, it is desirable that the area that cannot be observed by the camera as shown in Fig. 1 (b) is as small as possible.

If the reference weighted three-dimensional model is not recorded in the recording unit 190, the following processing is not executed. That is, when t ≦ t _s , the weighted three-dimensional model correction unit 142 outputs the weighted three-dimensional model generated in S130 as it is as a corrected weighted three-dimensional model. On the other hand, when t> t _s , the weighted three-dimensional model modification unit 142 outputs the weighted three-dimensional model modified by the method described below as a modified weighted three-dimensional model.

In the weighted three-dimensional model at time t, the number of voxels N _{i, t} contained in the part of the part identifier i shall satisfy N _{i, t} > N _i . Let V (i, x, t) represent the subset of the part of the part identifier i containing x voxels at time t. At this time, the subset V (i, N _i , t) that minimizes the error e _{i, t} at the part of the part identifier i at the time t calculated by the following equation is obtained.

Here, || · || represents the distance between the two position coordinates. Any distance can be used for the distance || · ||. Further, f (V (i, N _i , t)) represents the position of the three-dimensional center of gravity of the part of the subset V (i, N _i , t) calculated by the following equation.

Here, Σ on the left side of the above equation represents the sum when N _i integers are selected from {1,…, N _{i, t} }.

Any method may be used to obtain V (i, N _i , t) that minimizes the error e _{i, t} . The error e _{i, t} may be calculated for all combinations of N _i integers selected from {1,…, N _i } to determine the N _i voxels that take the minimum value. The error e _{i, t} may be calculated for a combination in which N _i integers are randomly selected for the number of times of, and the N _i voxels having the minimum value may be determined. The N _i voxels having the minimum value determined in this way are used as the site of the site identifier i again.

If N _{i, t} ≤ N _i , then N _{i, t} voxels are used as the site of the site identifier i.

Let the set of weighted voxel data corresponding to the voxels contained in the part of the part identifier i determined as described above be a corrected weighted three-dimensional model at time t.

The three-dimensional center of gravity position calculation unit 143 takes the corrected weighted three-dimensional model at time t generated in S142 as an input, calculates the three-dimensional center of gravity position at time t, and outputs it (S143). The three-dimensional center of gravity position COM (t) at time t is calculated by the following equation.

Here, N _all is the number of voxels included in the modified weighted 3D model generated in S142.

According to the invention of the present embodiment, it is possible to accurately estimate the position of the three-dimensional center of gravity of the human body from the multi-viewpoint image even when there is a region that cannot be observed by the camera.

<Supplementary note>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Communication unit, CPU (Central Processing Unit, cache memory, registers, etc.) to which can be connected, RAM and ROM as memory, external storage device as hard hardware, and input, output, and communication units of these. , CPU, RAM, ROM, has a bus connecting so that data can be exchanged between external storage devices. Further, if necessary, a device (drive) or the like capable of reading and writing a recording medium such as a CD-ROM may be provided in the hardware entity. As a physical entity equipped with such hardware resources, there is a general-purpose computer or the like.

The external storage device of the hardware entity stores a program required to realize the above-mentioned functions and data required for processing of this program (not limited to the external storage device, for example, reading a program). It may be stored in a ROM, which is a dedicated storage device). Further, the data obtained by the processing of these programs is appropriately stored in a RAM, an external storage device, or the like.

In the hardware entity, each program stored in the external storage device (or ROM, etc.) and the data required for processing of each program are read into the memory as needed, and are appropriately interpreted and executed and processed by the CPU. .. As a result, the CPU realizes a predetermined function (each configuration requirement represented by the above, ... Department, ... means, etc.).

The present invention is not limited to the above-described embodiment, and can be appropriately modified without departing from the spirit of the present invention. Further, the processes described in the above-described embodiment are not only executed in chronological order according to the order described, but may also be executed in parallel or individually depending on the processing capacity of the device that executes the processes or if necessary. ..

As described above, when the processing function in the hardware entity (device of the present invention) described in the above embodiment is realized by the computer, the processing content of the function that the hardware entity should have is described by the program. Then, by executing this program on the computer, the processing function in the above hardware entity is realized on the computer.

The program describing the processing content can be recorded on a computer-readable recording medium. The recording medium that can be read by a computer may be, for example, a magnetic recording device, an optical disk, a photomagnetic recording medium, a semiconductor memory, or the like. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) as an optical disk. Memory), CD-R (Recordable) / RW (ReWritable), etc., MO (Magneto-Optical disc), etc. as an optical magnetic recording medium, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. as a semiconductor memory. Can be used.

Further, the distribution of this program is performed, for example, by selling, transferring, renting, or the like a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in the storage device of the server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

A computer that executes such a program first temporarily stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, when the process is executed, the computer reads the program stored in its own storage device and executes the process according to the read program. Further, as another execution form of this program, a computer may read the program directly from a portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer. You may execute the process according to the received program one by one each time. In addition, the above-mentioned processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and the result acquisition without transferring the program from the server computer to this computer. May be. The program in this embodiment includes information used for processing by a computer and equivalent to the program (data that is not a direct command to the computer but has a property that regulates the processing of the computer, etc.).

Further, in this form, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of these processing contents may be realized in terms of hardware.

Claims (6)

Three-dimensional coordinates of the positions of voxels that form a three-dimensional area that represents the area occupied by the person's body at time t from multi-viewpoint images taken by multiple cameras installed so as to surround the person as the subject. A 3D model generator that generates a set of voxel data including at least voxel position coordinates as a 3D model at time t,
From the multi-viewpoint image, a three-dimensional joint position estimation unit that estimates three-dimensional coordinates representing the joint position of the person at time t as joint position coordinates at time t, and
From the three-dimensional model at the time t and the joint position coordinates at the time t, the part identifier for identifying the part of the human body including the voxel for the voxel constituting the three-dimensional region and the weight for the voxel are obtained. A weighted 3D model generator that generates a set of weighted voxel data including at least the voxel position coordinates, the site identifier, and the weight as a weighted 3D model at time t by determining.
The position of the three-dimensional center of gravity of the part is defined as the weighted average of the parts of the body of the person, and the position of the three-dimensional center of gravity is defined as the weighted average of the body of the person.
The weighted 3D model at time t, the part 3D center of gravity position of the part identifier i (i = 1, ..., N _p , N _p are the number of parts of the human body) at the time before time t and the predetermined time. A center of gravity position estimation device including a three-dimensional center of gravity position estimation unit that estimates the position of the three-dimensional center of gravity at time t from the weighted three-dimensional model (hereinafter referred to as a reference weighted three-dimensional model). The center of gravity position estimation device according to claim 1.
The three-dimensional center of gravity position estimation unit is
The part 3D center of gravity position estimation that estimates the part 3D center of gravity position of the part identifier i at time t from the part 3D center of gravity position of the part identifier i (i = 1, ..., N _p ) at the time before the time t. Department and
Corrected weight at time t from the part 3D center of gravity position of the part identifier i (i = 1, ..., N _p ) at time t, the reference weighted 3D model, and the weighted 3D model at time t. With a weighted 3D model modifier to generate a 3D model with
A center of gravity position estimation device including a three-dimensional center of gravity position calculation unit that calculates the three-dimensional center of gravity position at the time t from the corrected weighted three-dimensional model at the time t. The center of gravity position estimation device according to claim 2.
C3D (i, t) is the position of the part three-dimensional center of gravity of the part identifier i at the time t, N _i is the number of voxels contained in the part of the part identifier i in the reference weighted three-dimensional model, V (i, N _i) . Let t) be a subset of the part of the part identifier i containing N _i voxels at time t.
The weighted three-dimensional model correction part is
Find the subset V (i, N _i , t) that minimizes the error e _{i, t} at the part of the part identifier i at time t calculated by the following equation.

(However, f (V (i, N _i , t)) is the position of the three-dimensional center of gravity of the subset V (i, N _i , t).)
Generate a set of weighted voxel data of voxels contained in the minimized subset V (i, N _i , t) (i = 1, ..., N _p ) as a modified weighted 3D model at time t. A device for estimating the position of the center of gravity. The center of gravity position estimation device according to claim 2.
The site three-dimensional center of gravity position estimation unit is
By applying a time series filter to the part three-dimensional center of gravity position of the part identifier i (i = 1, ..., N _p ) at the time before the time t, the part three-dimensional of the part identifier i at the time t. A center of gravity position estimation device characterized by estimating the position of the center of gravity. A box cell that constitutes a three-dimensional area that represents the area occupied by the person's body at time t from multi-viewpoint images taken by a plurality of cameras installed so as to surround the person to be the subject. A three-dimensional model generation step that generates a set of boxel data including at least the boxel position coordinates, which are the three-dimensional coordinates of the position, as a three-dimensional model at time t.
A three-dimensional joint position estimation step in which the center of gravity position estimation device estimates three-dimensional coordinates representing the position of the person's joint at time t as joint position coordinates at time t from the multi-viewpoint image.
A part for the center of gravity position estimation device to identify a part of the human body including the voxel with respect to the voxel constituting the three-dimensional region from the three-dimensional model at the time t and the joint position coordinates at the time t. A weighted 3D model generation that generates a set of weighted voxel data including at least the boxel position coordinates, the part identifier, and the weight as a weighted 3D model at time t by determining the identifier and the weight for the boxel. Steps and
The position of the three-dimensional center of gravity of the part is defined as the weighted average of the parts of the body of the person, and the position of the three-dimensional center of gravity is defined as the weighted average of the body of the person.
The center of gravity position estimation device uses the weighted three-dimensional model at time t and the part tertiary of the part identifiers i (i = 1, ..., N _p , N _p are the number of parts of the human body) at times before time t. A method for estimating the position of the center of gravity, which includes a step of estimating the position of the center of gravity of the three dimensions at time t from the position of the original center of gravity and a weighted three-dimensional model at a predetermined time (hereinafter referred to as a reference weighted three-dimensional model). .. A program for operating a computer as the center of gravity position estimation device according to any one of claims 1 to 4.

Images