JPWO2020184066A1 - Joint position estimation device, joint position estimation method and program - Google Patents

Abstract

The joint position determination device 1 includes an image data acquisition unit 2 for acquiring image data, an estimation unit 3 for estimating a human body part from the acquired image data, a region of the human body part estimated by the estimation unit 3, and a human body part. For each area other than the area, the marking unit 4 that marks one or more pixels and the image data after marking by the marking unit 4 are applied with the water diversion ridge algorithm to make a plurality of areas in the image data. It includes a division unit 5 to be divided, and a calculation unit 6 to calculate a joint position from two areas corresponding to human body parts adjacent to the joint among the areas divided by the division unit 5.

Description

The present invention relates to a joint position estimation device and a joint position estimation method for estimating a joint position from image data, and further relates to a computer-readable recording medium in which a program for realizing these is stored.

Conventionally, various techniques have been proposed in which a person is imaged with an imaging device and the posture of the person and the movement (gesture, etc.) performed by the person are estimated. For example, Non-Patent Document 1 discloses a technique of estimating the joint position of a person and estimating the posture of the person. In Non-Patent Document 1, a human body part of a person is estimated from a depth image, and the joint position is estimated by a joint position estimator using the estimation result.

Ryuhei Takahashi, "Special Research Report CNN's Estimating Human Posture with Invariant Perspective Using Depth Images", February 8, 2018, Internet <URL: http: //www.mm.media.kyoto-u.ac .jp / wp-content / uploads / 2018/02/2017-b-takahashi.pdf>

However, Non-Patent Document 1 has a problem that the estimation accuracy of the joint position is lowered unless the human body part is estimated accurately.

An example of an object of the present invention is to provide a joint position estimation device, a joint position estimation method, and a computer-readable recording medium capable of accurately estimating a joint position.

In order to achieve the above object, the joint position estimation device in one aspect of the present invention is used.
The image data acquisition unit that acquires image data and
An estimation unit that estimates a human body part from the image data acquired by the image data acquisition unit, and an estimation unit.
When the estimation unit estimates a plurality of human body parts, a marking unit that marks one or a plurality of pixels for each of the region of the human body part estimated by the estimation unit and the region other than the region of the human body part. ,
A watershed algorithm is applied to the image data after marking by the marking unit to divide the area in the image data into a plurality of divided units.
A calculation unit that calculates the joint position from two regions corresponding to the human body part adjacent to the joint among the regions divided by the division portion.
To be equipped.

Further, in order to achieve the above object, the joint position estimation method in one aspect of the present invention is:
Steps to get image data and
A step of estimating a human body part based on the image data,
When a plurality of human body parts are estimated from the image data, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body part region.
A step of applying a watershed algorithm to the image data after marking to divide the area in the image data into a plurality of areas, and
A step of calculating the joint position from two regions corresponding to the human body part adjacent to the joint among the divided regions, and
To be equipped.

Further, in order to achieve the above object, the computer-readable recording medium in one aspect of the present invention is used in a computer for acquiring image data.
A step of estimating a human body part based on the image data,
When a plurality of human body parts are estimated, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body region.
A step of applying a watershed algorithm to the image data after marking to divide the area in the image data into a plurality of areas, and
A step of calculating the joint position from two regions corresponding to the human body part adjacent to the joint among the divided regions, and
The program is recorded, including the instruction to execute.

As described above, according to the present invention, the joint position can be estimated accurately.

FIG. 1 is a block diagram showing a schematic configuration of the joint position estimation device according to the first embodiment. FIG. 2 is a block diagram specifically showing the configuration of the joint position estimation device. FIG. 3 is a diagram showing an example of image data. FIG. 4 is a diagram showing a feature amount obtained by the feature vector calculation unit. FIG. 5 is a diagram showing a region set for the estimation result by the estimation unit shown in FIG. FIG. 6 is a diagram showing a marking result by the marking unit. FIG. 7 is a diagram showing the result of region division by the division portion. FIG. 8 is a diagram for explaining a calculation method by the calculation unit. FIG. 9 is a flow chart showing the operation of the joint position estimation device. FIG. 10 is a block diagram showing an example of a computer that realizes the joint position estimation device according to the embodiment.

Hereinafter, the joint position estimation device and the joint position estimation method according to the embodiment of the present invention will be described with reference to FIGS. 1 to 10.

[Device configuration]
FIG. 1 is a block diagram showing a schematic configuration of the joint position estimation device 1 according to the first embodiment.

The joint position estimation device 1 is a device that estimates parts of the human body (for example, palm and forearm) from image data and calculates the position of joints from the estimated human body parts. The joint position estimation device 1 includes an image data acquisition unit 2, an estimation unit 3, a marking unit 4, a division unit 5, and a calculation unit 6.

The image data acquisition unit 2 acquires image data.

The estimation unit 3 estimates a human body part from the image data acquired by the image data acquisition unit 2.

When the estimation unit 3 estimates a plurality of human body parts, the marking unit 4 has one or a plurality of pixels for each of the region of the human body part estimated by the estimation unit 3 and the region other than the region of the human body part. Mark.

The dividing unit 5 applies the watershed algorithm to the image data after marking by the marking unit 4 to divide the area in the image data into a plurality of areas.

The calculation unit 6 calculates the joint position from two regions corresponding to the human body portion adjacent to the joint among the regions divided by the division unit 5.

As described above, in the present embodiment, the estimation result of the human body part is marked and the watershed algorithm is applied. Therefore, the marking can be performed with high accuracy, and the area division of the image data can be performed with high accuracy by applying the watershed algorithm with the result as an input. As a result, the region of the human body adjacent to the joint can be estimated accurately, and a highly reliable calculation result of the joint (wrist) position can be obtained.

Subsequently, with reference to FIG. 2, the configuration of the joint position estimation device 1 in the present embodiment will be described more specifically. FIG. 2 is a block diagram specifically showing the configuration of the joint position estimation device 1.

The joint position estimation device 1 includes an area setting unit 7 in addition to the image data acquisition unit 2, the estimation unit 3, the marking unit 4, the division unit 5, and the calculation unit 6 described above.

The image data acquisition unit 2 acquires the image data 50. The image data acquisition unit 2 may acquire the image data 50 via a network such as a wired or wireless LAN (Local Area Network), or may acquire the image data 50 by short-range wireless communication or the like. Further, an image pickup device may be directly connected to the joint position estimation device 1, or the joint position estimation device 1 may include an image pickup device, and the image data acquisition unit 2 may acquire image data 50 from the image pickup device.

FIG. 3 is a diagram showing an example of image data 50. The image data 50 is an image captured from the forearm to the finger of the human body. The image data 50 in this embodiment is image data to which a depth for each pixel is added (hereinafter, referred to as a TOF image). The TOF image is an image in which information (depth information) indicating the distance from the imaging device to the imaging target is associated with each pixel. The image data 50 is not limited to the TOF image, and may be a two-dimensional image.

Further, the image data 50 may be an image in which the entire image of the person is captured. In this case, for example, the image data acquisition unit 2 may estimate a portion including a joint (for example, the entire hand) from an image in which the entire image of the person is captured, and extract image data of that portion. Then, the subsequent processing may be performed on the extracted image data.

The estimation unit 3 estimates the type (forearm, finger, etc.) of the human body part shown in the image data 50. The estimation unit 3 has a feature vector calculation unit 31 and an estimation execution unit 32.

The feature vector calculation unit 31 obtains a feature amount (feature vector) for each pixel of the image data 50. FIG. 4 is a diagram showing a feature amount obtained by the feature vector calculation unit 31. As shown in FIG. 4, the feature vector calculation unit 31 obtains each feature amount 51, 52, 53, 54. The estimation execution unit 32 performs machine learning by a neural network or the like, and uses the learning model 60 to estimate the type of the object shown in the image data 50 from the feature amount obtained by the feature vector calculation unit 31. In the case of FIG. 4, the estimation execution unit 32 estimates that the feature amount 51 is the feature amount of the forearm, the feature amount 52 is the feature amount of the palm, and the feature amount 53 is the feature amount of the finger. Further, the estimation execution unit 32 estimates that the feature amount 54 is a feature amount of an object reflected in the background other than the human body part. In FIG. 4, the human body part shown in FIG. 3 is shown by a broken line in order to understand the feature amount obtained by the feature vector calculation unit 31.

Since the image data 50 is a TOF image, the estimation accuracy of the human body part by the estimation unit 3 is high. For example, when the image data 50 is an image including joints, the feature vector calculation unit 31 can accurately obtain the feature amount by using the depth information for each pixel even if the joints are in a bent posture. Can be done. As a result, the estimation accuracy of the human body part by the estimation execution unit 32 becomes high.

The area setting unit 7 sets a region including the forearm and the palm, which are human body parts adjacent to the joints, among the human body parts estimated by the estimation unit 3. FIG. 5 is a diagram showing a region 55 set for the estimation result by the estimation unit 3 shown in FIG. In FIG. 5, the region 55 includes the finger feature 53, but at least the forearm feature 51 adjacent to the joint (wrist) and the palm feature 52 may be included. By setting the area 55, it is possible to prevent the unnecessary area from being processed in the subsequent processing.

The marking unit 4 marks one or more pixels in the area 55 set by the area setting unit 7. FIG. 6 is a diagram showing a marking result by the marking unit 4. The marking portion 4 initially marks the human body part (forearm, palm) adjacent to the joint (wrist) and the area other than the human body part (background) in the area 55. The marking 56 shown in FIG. 6 is an initial marking set for the feature amount 51 of the forearm. The marking 57 is an initial marking set for the feature amount 52 of the palm. The marking 58 indicates an initial marking set for the background area.

When the marking portion 4 initially marks the feature amount 51 of the forearm and the feature amount 52 of the palm, noise affects when it is close to the boundary with other regions. Therefore, in order to prevent the influence of noise, each feature is used. Mark the center of the area of quantity. Further, when the marking portion 4 initially marks the background area, the marking portion 4 marks a position outside a certain distance range from the markings 56 and 57. The marking unit 4 may mark only one pixel, but in the watershed algorithm described later, if the change in the depth value of the TOF image is not smooth, the marked area may not be widened. Therefore, it is preferable that the marking unit 4 marks a plurality of pixels.

The dividing unit 5 applies a watershed algorithm that inputs markings 56, 57, and 58 set by the marking unit 4 to divide the area 55 set by the area setting unit 7 into a plurality of areas. In the watershed algorithm, the markings 56, 57, and 58 are set as the minimum value (or maximum value) of the pixel value (depth value), and the point where the maximum value (or minimum value) of the pixel value is obtained while expanding the area is obtained. A region where the pixel values (depth values) are close to each other is defined as one region with the point at which the maximum value is the boundary as a boundary. FIG. 7 is a diagram showing the result of region division by the division unit 5. In FIG. 7, the region 55 is divided into a forearm region 59A, a palm region 59B, and a background region 59C.

The watershed algorithm is not necessarily divided into the same number of regions as the number of markings, and may be divided into four or more regions in FIG. 7. Further, although the division portion 5 expands the marked area using the depth value, the luminance value or the like may be used as long as it is the information of each pixel of the image data 50.

The calculation unit 6 calculates the joint position from the two regions 59A and 59B corresponding to the human body part among the regions divided by the division unit 5. FIG. 8 is a diagram for explaining a calculation method by the calculation unit 6. The calculation unit 6 performs labeling processing on each of the areas 59A and 59B, and acquires the contour of each area. In FIG. 8, the contour is shown schematically. As shown in FIG. 8, when the area 59A and the area 59B are separated from each other, the calculation unit 6 obtains the coordinate positions 59A1 and 59B1 closest to each other in the two areas 59A and 59B. The calculation unit 6 calculates a region 59D including the shortest straight line L connecting the coordinate positions 59A1 and 59B1 to obtain the joint position. When the contours of the two regions 59A and 59B overlap, the calculation unit 6 may use the overlapping portion as the joint position.

[Operation description]
Next, the operation of the joint position determination device in the present embodiment will be described with reference to FIG. FIG. 9 is a flow chart showing the operation of the joint position estimation device 1. In the following description, FIGS. 1 to 8 will be referred to as appropriate. Further, in the present embodiment, the joint position estimation method is carried out by operating the joint position estimation device. Therefore, the description of the joint position estimation method in the present embodiment is replaced with the following operation description of the joint position estimation device.

First, as a premise, it is assumed that a human body part including a joint is imaged by an imaging device and the image data is input to the joint position estimation device 1.

Under the above premise, the image data acquisition unit 2 acquires the input image data (S1). Next, the feature vector calculation unit 31 of the estimation unit 3 obtains the feature amounts 51 to 54 (see FIG. 4) for each pixel of the image data 50 (S2). Then, the estimation execution unit 32 of the estimation unit 3 recognizes the type of the object having the feature amount from the feature amount obtained by the feature vector calculation unit 31, and estimates the human body part in the image data 50 (S3).

Subsequently, the region setting unit 7 sets a region 55 (see FIG. 5) adjacent to the joint whose position is to be estimated and which includes the human body portion estimated by the estimation unit 3 (S4). The marking unit 4 initially marks in the area 55 set by the area setting unit 7 (S5). In S5, the marking portion 4 provides a marking 56 for the forearm (see FIG. 6) adjacent to the joint (wrist) to be estimated for position, a marking 57 for the palm, and a marking 58 for an area (background) other than the human body part. Set.

The dividing unit 5 applies a watershed algorithm that inputs markings 56, 57, and 58 to divide the region 55 set in S4 into a plurality of regions 59A, 59B, and 59C (see FIG. 7) (S6). Then, the calculation unit 6 calculates the joint position from the two regions 59A and 59B corresponding to the human body portion among the regions divided by the division unit 5 (S7).

[Effect of this embodiment]
As described above, according to the present embodiment, the human body part is estimated from the image data 50, and the estimation result is initially marked. Therefore, in the image data 50, even if the target area to be marked, for example, the shape of the forearm or the palm is a complicated shape, the initial marking can be performed with high accuracy. By using the watershed algorithm with the initial marking as an input, the estimation accuracy of the two regions adjacent to the joint, the forearm and the palm, is improved. If one area of the forearm or palm cannot be clearly determined, the position of the joint (wrist) between the two areas cannot be calculated accurately, but by estimating the two areas accurately, a highly reliable joint can be calculated. The calculation result of the position of (wrist) can be obtained.

Further, once the position of the joint, for example, the wrist, is determined, it becomes easy to estimate the position of the finger at the tip of the wrist, and it becomes easy to estimate the movement of the hand and the posture of the hand. As a result, it can be applied to a device that recognizes a gesture of moving a hand.

[program]
The program according to the embodiment of the present invention may be any program that causes a computer to execute steps S1 to S7 shown in FIG. By installing this program on a computer and executing it, the joint position estimation device and the joint position estimation method according to the present embodiment can be realized. In this case, the computer processor functions as an image data acquisition unit 2, an estimation unit 3, a marking unit 4, a division unit 5, and a calculation unit 6 to perform processing.

Further, in the present embodiment, the image data acquisition unit 2, the estimation unit 3, the marking unit 4, the division unit 5, and the calculation unit 6 store the data files constituting them in a storage device such as a hard disk provided in the computer. This is achieved by doing so or by mounting the recording medium in which this data file is stored in a reader connected to a computer.

Moreover, the program in this embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as any of the image data acquisition unit 2, the estimation unit 3, the marking unit 4, the division unit 5, and the calculation unit 6, respectively.

[Physical configuration]
Here, a computer that realizes the joint position estimation device 1 by executing the program in the present embodiment will be described with reference to FIG. FIG. 10 is a block diagram showing an example of a computer that realizes the joint position estimation device 1 according to the present embodiment.

As shown in FIG. 10, the computer 110 includes a CPU 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. Each of these parts is connected to each other via a bus 121 so as to be capable of data communication. The computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to the CPU 111 or in place of the CPU 111.

The CPU 111 expands the programs (codes) of the present embodiment stored in the storage device 113 into the main memory 112 and executes them in a predetermined order to perform various operations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Further, the program in the present embodiment is provided in a state of being stored in a computer-readable recording medium 120. The program in the present embodiment may be distributed on the Internet connected via the communication interface 117.

Further, specific examples of the storage device 113 include a semiconductor storage device such as a flash memory in addition to a hard disk drive. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads a program from the recording medium 120, and writes a processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.

Specific examples of the recording medium 120 include a general-purpose semiconductor storage device such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), a magnetic recording medium such as a flexible disk, or a CD-. Examples include optical recording media such as ROM (Compact Disk Read Only Memory).

The joint position estimation device 1 in the present embodiment can also be realized by using hardware corresponding to each part instead of the computer in which the program is installed. Further, the joint position estimation device 1 may be partially realized by a program and the rest may be realized by hardware.

A part or all of the above-described embodiments can be expressed by the following descriptions (Appendix 1) to (Appendix 15), but the present invention is not limited to the following description.

(Appendix 1)
The image data acquisition unit that acquires image data and
An estimation unit that estimates a human body part from the image data acquired by the image data acquisition unit, and an estimation unit.
When the estimation unit estimates a plurality of human body parts, a marking unit that marks one or a plurality of pixels for each of the region of the human body part estimated by the estimation unit and the region other than the region of the human body part. ,
A watershed algorithm is applied to the image data after marking by the marking unit to divide the area in the image data into a plurality of divided units.
A calculation unit that calculates the joint position from two regions corresponding to the human body part adjacent to the joint among the regions divided by the division portion.
A joint position estimation device.

(Appendix 2)
The joint position estimation device according to Appendix 1.
An area setting unit that sets an area adjacent to a joint that includes a human body part estimated by the estimation unit,
With more
The marking unit marks within the area set by the area setting unit.
The division unit divides the area set by the area setting unit into a plurality of areas.
Joint position estimation device.

(Appendix 3)
The joint position estimation device according to Appendix 1 or Appendix 2.
The image data is image data to which a depth for each pixel is added.
Joint position estimation device.

(Appendix 4)
The joint position estimation device according to any one of Supplementary note 1 to Supplementary note 3.
The dividing portion divides the region into a plurality of regions, with the pixels marked by the marking portion and the pixels whose pixel values are within a predetermined range of the pixels as the same region.
Joint position estimation device.

(Appendix 5)
The joint position estimation device according to any one of Supplementary note 1 to Supplementary note 4.
The calculation unit calculates the joint position with reference to the position in each of the two regions where the minimum distance between the two regions is the minimum distance.
Joint position estimation device.

(Appendix 6)
Steps to get image data and
Steps to estimate the human body part from the acquired image data,
When a plurality of human body parts are estimated, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body region.
A step of applying a watershed algorithm to the image data after marking to divide the area in the image data into a plurality of areas, and
A step of calculating the joint position from two regions corresponding to the human body part adjacent to the joint among the divided regions, and
A method for estimating joint position.

(Appendix 7)
The joint position estimation method described in Appendix 6
Steps to set the area adjacent to the joint and containing the estimated human body part,
With more
In the step of marking one or more pixels, marking is performed within the set area.
In the step of dividing the area in the image data into a plurality of areas, the set area is divided into a plurality of areas.
Joint position estimation method.

(Appendix 8)
The joint position estimation method according to Appendix 6 or Appendix 7, wherein the joint position is estimated.
The image data is image data to which a depth for each pixel is added.
Joint position estimation method.

(Appendix 9)
The joint position estimation method according to any one of Supplementary note 6 to Supplementary note 8.
In the step of dividing the area in the image data into a plurality of areas, the marked pixel and the pixel whose pixel value is within a predetermined range of the pixel are set as the same area, and the area is divided into a plurality of areas.
Joint position estimation method.

(Appendix 10)
The joint position estimation method according to any one of Supplementary note 6 to Supplementary note 9.
In the step of calculating the joint position, the joint position is calculated with reference to the position in each of the two regions where the distance between the two regions is the minimum distance.
Joint position estimation method.

(Appendix 11)
On the computer that acquires the image data,
A step of estimating a human body part based on the image data,
When a plurality of human body parts are estimated, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body region.
A step of applying a watershed algorithm to the image data after marking to divide the area in the image data into a plurality of areas, and
A step of calculating the joint position from two regions corresponding to the human body part adjacent to the joint among the divided regions, and
A computer-readable recording medium on which a program is recorded, including instructions to execute.

(Appendix 12)
The computer-readable recording medium according to Appendix 11, which is a computer-readable recording medium.
The program
On the computer
Steps to set the area adjacent to the joint and containing the estimated human body part,
Including further instructions to execute
In the step of marking one or more pixels, marking is performed within a set area.
In the step of dividing the area in the image data into a plurality of areas, the set area is divided into a plurality of areas.
A computer-readable recording medium.

(Appendix 13)
The computer-readable recording medium according to Appendix 11 or Appendix 12, wherein the recording medium is readable.
The image data is image data to which a depth for each pixel is added.
A computer-readable recording medium.

(Appendix 14)
The computer-readable recording medium according to any one of Supplementary note 11 to Supplementary note 13.
In the step of dividing the area in the image data into a plurality of areas, the marked pixel and the pixel whose pixel value is within a predetermined range of the pixel are set as the same area, and the area is divided into a plurality of areas.
A computer-readable recording medium.

(Appendix 15)
The computer-readable recording medium according to any one of Supplementary note 11 to Supplementary note 14.
In the step of calculating the joint position, the joint position is calculated with reference to the position in each of the two regions where the distance between the two regions is the minimum distance.
A computer-readable recording medium.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the structure and details of the present invention.

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2019-46390 filed on March 13, 2019, and the entire disclosure thereof is incorporated herein by reference.

The present invention can accurately estimate the joint position.

1: Joint position estimation device 2: Image data acquisition unit 3: Estimate unit 4: Marking unit 5: Division unit 6: Calculation unit 7: Area setting unit 31: Feature vector calculation unit 32: Estimation execution unit 50: Image data 60: Learning models 51, 52, 53, 54: Feature 55: Areas 56, 57, 58: Marking

The present invention relates to a joint position estimation device and joint position estimating method for estimating a joint position from the image data, and further relates to a program for realizing these.

An example of an object of the present invention is to provide a joint position estimation device, a joint position estimation method, and a program capable of accurately estimating a joint position.

Further, in order to achieve the above object, the program in one aspect of the present invention is applied to a computer that acquires image data.
A step of estimating a human body part based on the image data,
When a plurality of human body parts are estimated, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body region.
A step of applying a watershed algorithm to the image data after marking to divide the area in the image data into a plurality of areas, and
A step of calculating the joint position from two regions corresponding to the human body part adjacent to the joint among the divided regions, and
Including an instruction to the execution.

(Appendix 11)
On the computer that acquires the image data,
A step of estimating a human body part based on the image data,
When a plurality of human body parts are estimated, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body region.
A step of applying a watershed algorithm to the image data after marking to divide the area in the image data into a plurality of areas, and
A step of calculating the joint position from two regions corresponding to the human body part adjacent to the joint among the divided regions, and
The including the instruction to be executed, program.

(Appendix 12)
The program described in Appendix 11
Before Symbol computer,
Steps to set the area adjacent to the joint and containing the estimated human body part,
Including further instructions to execute
In the step of marking one or more pixels, marking is performed within a set area.
In the step of dividing the area in the image data into a plurality of areas, the set area is divided into a plurality of areas.
Program .

(Appendix 13)
The program according to Appendix 11 or Appendix 12, which is described in Appendix 12.
The image data is image data to which a depth for each pixel is added.
Program .

(Appendix 14)
The program described in any one of Supplementary Note 11 to Supplementary Note 13.
In the step of dividing the area in the image data into a plurality of areas, the marked pixel and the pixel whose pixel value is within a predetermined range of the pixel are set as the same area, and the area is divided into a plurality of areas.
Program .

(Appendix 15)
The program described in any one of Supplementary note 11 to Supplementary note 14.
In the step of calculating the joint position, the joint position is calculated with reference to the position in each of the two regions where the distance between the two regions is the minimum distance.
Program .

Claims (15)

Image data acquisition means for acquiring image data and
An estimation means for estimating a human body part from the image data acquired by the image data acquisition means, and an estimation means.
When the estimation means estimates a plurality of human body parts, the marking means for marking one or a plurality of pixels for each of the region of the human body part estimated by the estimation means and the region other than the region of the human body part. ,
A watershed algorithm is applied to the image data after marking by the marking means to divide the area in the image data into a plurality of parts.
Of the regions divided by the dividing means, a calculation means for calculating the joint position from two regions corresponding to the human body part adjacent to the joint, and
A joint position estimation device. The joint position estimation device according to claim 1.
A region setting means that sets a region adjacent to a joint and includes a human body part estimated by the estimation means,
With more
The marking means marks within the area set by the area setting means.
The dividing means divides the area set by the area setting means into a plurality of areas.
Joint position estimation device. The joint position estimation device according to claim 1 or 2.
The image data is image data to which a depth for each pixel is added.
Joint position estimation device. The joint position estimation device according to any one of claims 1 to 3.
The dividing means divides a region into a plurality of regions with the pixels marked by the marking means and the pixels whose pixel values are within a predetermined range of the pixels as the same region.
Joint position estimation device. The joint position estimation device according to any one of claims 1 to 4.
The calculation means calculates the joint position with reference to the position in each of the two regions where the minimum distance between the two regions is the minimum distance.
Joint position estimation device. Get image data,
The human body part is estimated from the image data, and
When a plurality of human body parts are estimated with respect to the image data, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body part region.
A watershed algorithm is applied to the image data after marking to divide the area in the image data into a plurality of areas.
The joint position is calculated from two regions corresponding to the human body part adjacent to the joint among the plurality of divided regions.
Joint position estimation method. The joint position estimation method according to claim 6.
Of the human body parts estimated from the image data, a region including the human body part adjacent to the joint is set.
When marking the one or more pixels, mark within the area including the human body part adjacent to the joint.
A joint position estimation method for dividing a region including a human body part adjacent to a joint into a plurality of regions when the region in the image data is divided into a plurality of regions. The joint position estimation method according to claim 6 or 7.
The image data is image data to which a depth for each pixel is added.
Joint position estimation method. The joint position estimation method according to any one of claims 6 to 8.
When the area in the image data is divided into a plurality of areas, the area is divided into a plurality of areas with the marked one or a plurality of pixels and the pixels whose pixel values are within a predetermined range of the pixels as the same area.
Joint position estimation method. The joint position estimation method according to any one of claims 6 to 9.
When calculating the joint position, the joint position is calculated based on the position in each of the two regions where the minimum distance between the two regions is the minimum distance.
Joint position estimation method. On the computer that acquires the image data,
The human body part is estimated from the image data, and
When a plurality of human body parts are estimated, one or a plurality of pixels are marked for each of the estimated human body part region and the region other than the human body region.
A watershed algorithm is applied to the image data after the marking to divide the area in the image data into a plurality of areas.
Of the divided regions, the joint position is calculated from the two regions corresponding to the human body parts adjacent to the joints.
A computer-readable recording medium that records a program that executes an instruction. The computer-readable recording medium according to claim 11.
The program
On the computer
Have the area set up adjacent to the joint and containing the estimated human body part,
Including more instructions
When marking one or more pixels, mark within the set area.
When the area in the image data is divided into a plurality of areas, the set area is divided into a plurality of areas.
A computer-readable recording medium. The computer-readable recording medium according to claim 11 or 12.
The image data is image data to which a depth for each pixel is added.
A computer-readable recording medium. The computer-readable recording medium according to any one of claims 11 to 13.
When the area in the image data is divided into a plurality of areas, the marked pixel and the pixel whose pixel value is within a predetermined range of the pixel are set as the same area, and the area is divided into a plurality of areas.
A computer-readable recording medium. The computer-readable recording medium according to any one of claims 11 to 14.
When calculating the joint position, the joint position is calculated based on the position in each of the two regions where the minimum distance between the two regions is the minimum distance.
A computer-readable recording medium.

Images