JP7447956B2 - Processing device, attitude analysis system, and program - Google Patents

Description

The present invention relates to a processing device, a posture analysis system, and a program.

As a technology for detecting an object, for example, there is a technology described in Patent Document 1 below, but it is desired that the technology for detecting an object detects the shape or posture of all or part of the object with high precision. ing.

Japanese Patent Application Publication No. 2007-288682

According to an aspect of the present invention, the acquisition unit that acquires the first point cloud data determines the principal component direction by performing principal component analysis on the first point cloud data, and determines the principal component direction at a plurality of positions in the principal component direction. a calculation unit that calculates the feature amount of the shape indicated by the second point cloud data obtained by extracting each point existing in the area intersecting the direction; and a determination unit that determines the change in the feature amount at a plurality of positions in the principal component direction. A processing device is provided.

According to an aspect of the present invention, there is provided an attitude analysis system including the processing device of the above aspect and a detection unit that detects distance information to an object and generates point cloud data from the distance information.

According to an aspect of the present invention, there is provided a program that is executed by a computer, which acquires first point cloud data, determines a principal component direction by principal component analysis of the first point cloud data, and determines a plurality of principal component directions. At the position, calculate the feature amount of the shape indicated by the second point cloud data extracted from each point existing in the area intersecting the principal component direction, and determine the change in the feature amount at multiple positions in the principal component direction. , a program for executing the process is provided.

FIG. 1 is a diagram showing an example of a processing device (posture analysis system) according to a first embodiment. FIG. 3 is a diagram illustrating an example of a position detection unit (point cloud acquisition unit) according to the present embodiment. FIG. 2 is a functional block diagram showing an example of the processing device shown in FIG. 1. FIG. FIG. 3 is a diagram in which a first region is set for the reference point group data according to the present embodiment. FIG. 6 is a diagram in which the first principal component is calculated from the first point cloud data of the first region and the second region is set according to the present embodiment. It is a figure which shows an example of calculating the shortest path about the 2nd point group data based on this embodiment. It is a figure which shows the other example which calculated the shortest path about the 2nd point cloud data based on this embodiment. FIG. 3 is a graph diagram showing the relationship between the position in the direction of the first principal component and the shortest path length according to the present embodiment. It is a diagram in which a second area is set from a new first area according to the present embodiment. FIG. 7 is a graph diagram showing the relationship between the position in the direction of the first principal component and the shortest path length for a new second region according to the present embodiment. (A) and (B) are diagrams illustrating an example of estimating the posture of an object by the second generation unit according to the present embodiment. 3 is a flowchart illustrating an example of a processing method according to the present embodiment. 7 is a flowchart showing another example of the processing method according to the present embodiment. 14 is a flowchart showing another example of the processing method following FIG. 13. FIG. It is a figure showing an example of a processing device concerning a 2nd embodiment. It is a figure showing an example of a processing device concerning a 3rd embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment. It is a figure showing an example of the screen displayed on the display part concerning this embodiment.

Hereinafter, embodiments will be described with reference to the drawings. In the drawings, in order to explain the embodiments, the scale of the drawings is changed as appropriate, such as by enlarging or emphasizing some parts. In the drawings, there are diagrams for explaining directions in the drawings using an XYZ coordinate system. In this XYZ coordinate system, a plane parallel to the horizontal plane is defined as an XZ plane. In this XZ plane, the direction of the shoulder width of the human body M is referred to as the X direction, and the direction perpendicular to the X direction is referred to as the Z direction. The direction perpendicular to the XZ plane is expressed as the Y direction. The X direction, Y direction, and Z direction will be described assuming that the direction indicated by the arrow in the figure is the + direction, and the direction opposite to the direction indicated by the arrow is the - direction.

[First embodiment]
A first embodiment will be described. FIG. 1 is a diagram showing an example of a processing device 1A according to the first embodiment. FIG. 1 also shows an example of the posture analysis system SYS according to the embodiment. The processing device 1A and the posture analysis system SYS generate one or both of posture information and shape information of the object M. In this embodiment, a human body will be described as an example of the object M, but the object M may be, for example, an animal, a humanoid or a non-human animal type robot, or a non-animal type robot. Hereinafter, the object M will be described as a human body M.

The human body M is in a predetermined static posture (e.g., a specified pose, a standing state) in a target area AR (e.g., detection area of the position detection unit 2, field of view) that is a detection target of the position detection unit 2, which will be described later. It may be in the state of , or it may be in the state of moving (eg, changing, moving). For example, the human body M changes one or both of its posture and shape as it moves. The moving human body M is, for example, a sport such as ballet, fencing, baseball, soccer, golf, kendo, American football, ice hockey, or gymnastics, walking or posing, such as running, exercise, yoga, body building, a fashion show, or a game. , training, dance, traditional performing arts, person recognition, or the human body moving during work. The human body M may include other objects attached to the human body M (eg, clothing, attachments, exercise equipment, protective equipment).

The processing device 1A (or posture analysis system SYS) is, for example, a shape evaluation device, a motion detection device, a motion evaluation device, a motion capture device, a motion detection system, an exercise support system, or the like. Posture analysis system SYS includes a processing device 1A and a position detection section 2. The position detection unit 2 is, for example, a point cloud acquisition unit. The posture analysis system SYS may include a plurality of processing devices 1A or a plurality of position detection units 2 (eg, point cloud acquisition units). The processing device 1A (or attitude analysis system SYS) of this embodiment includes two position detection units 2. Note that although the two position detection units 2 have the same configuration, they may have different configurations. One of the two position detection units 2 has a viewpoint set on the front side of the human body M (eg, +Z side), and the other has a viewpoint set on the back side of the human body M (eg, -Z side). The two position detection units 2 may be arranged symmetrically with respect to the human body M, or may be arranged asymmetrically. Then, the processing device 1A (or posture analysis system SYS) generates three-dimensional data that is information about the shape or posture of the human body M based on the reference point group data PF obtained from the surface of the human body M. The reference point group data PF is information representing the shape (eg, three-dimensional shape) of the human body M.

The processing device 1A includes a generation section 11, a first generation section 12, a second generation section 13, a calculation section 14, and a storage section 15, as shown in FIG. Part or all of the processing device 1A may be a portable device (eg, an information terminal, a smartphone, a tablet, a mobile phone with a camera, a wearable terminal). Further, part or all of the processing device 1A may be a stationary device (eg, a desktop computer).

Next, each part constituting the processing device 1A will be explained. The generation unit 11 extracts each point within the first region R1 (for example, see FIG. 5) from the reference point group data PF (for example, see FIG. 4), which is positional information of each point on the human body M, and One point group data P1 is generated. The calculation unit 14 calculates the direction of the first principal component D1 (eg, see FIG. 5) in the first point cloud data P1 by principal component analysis of the first point cloud data P1 generated by the generation unit 11. Then, the first generation unit 12 generates a second area having a plane RS2A (for example, see FIG. 5), which is a plane intersecting the direction of the first principal component D1 calculated by the calculation unit 14, from the first point group data P1. Points within R2 (eg, see FIG. 5) are extracted to generate second point group data P2. The second generation unit 13 generates information about the human body M using the second point group data P2 generated by the first generation unit 12. Details of the generation section 11, first generation section 12, second generation section 13, and calculation section 14 will be described later. Note that the first area R1 shown in FIG. 4 is a smaller area than the reference area formed by the reference point group data PF.

The storage unit 15 is, for example, a volatile or nonvolatile memory, a hard disk (HDD), a solid state drive (SSD), or the like. The processing device 1A (or posture analysis system SYS) includes a display section 3 and an input section 4. The display unit 3 displays this image based on the image data output from the processing device 1A. For example, the processing device 1A outputs data of a model image related to the human body M or an estimated image related to the human body M generated by the second generation unit 13 to the display unit 3. The display unit 3 then displays the estimated image based on the estimated image data output from the processing device 1A. The display unit 3 includes, for example, a liquid crystal display or an organic EL display.

The input unit 4 is used to input various information (eg, data, commands) to the processing device 1A. By operating the input unit 4, the user can input various information to the processing device 1A. The input unit 4 includes, for example, at least one of a keyboard, a mouse, a trackball, a touch pad, and an audio input device (eg, a microphone). Note that the display section 3 and the input section 4 may be touch panels. The processing device 1A (or the attitude analysis system SYS) does not need to include the display unit 3. For example, the processing device 1A may output data such as an image to another display unit or display device via communication, and this display unit or the like may display the image. Further, the processing device 1A (posture analysis system SYS) does not need to include the input section 4. For example, the processing device 1A may be configured to receive various commands and information via communication.

The position detection unit 2 detects position information of the human body M. The position detection unit 2 detects point cloud data including three-dimensional coordinates of each point on the surface of the human body M as position information of the human body M. As shown in FIG. 1, the position detection section 2 includes a detection section 21, a reference point group data generation section 22, and a storage section 23. The storage unit 23 is, for example, a volatile or nonvolatile memory, a hard disk (HDD), a solid state drive (SSD), or the like. Note that the position detection section 2 does not need to include the storage section 23.

The detection unit 21 is, for example, at least a part of a portable device (portable device). The detection unit 21 may be at least a part of a stationary device. The reference point group data generation unit 22 may be provided in a device external to the processing device 1A (eg, may be externally connected). For example, the position detection unit 2 may be a device external to the processing device 1A, and may include a detection unit 21, a reference point group data generation unit 22, and a storage unit 23. Part or all of the position detection unit 2 may be a portable device (eg, an information terminal, a smartphone, a tablet, a mobile phone with a camera, a wearable terminal). Further, a part or the whole of the position detection unit 2 may be a stationary device (eg, a fixed point camera).

FIG. 2 is a diagram showing an example of the detection unit 21 of the position detection unit 2 (eg, point cloud acquisition unit). The detection unit 21 detects the depth as position information of the human body M. The detection unit 21 includes a depth sensor (eg, a depth camera). The detection unit 21 detects the depth (eg, position information, distance, depth, depth) from a predetermined point to each point on the surface of the human body M placed in the target region AR. The predetermined point is, for example, a point at a position that serves as a reference for detection by the detection unit 21 (e.g., a viewpoint, a detection source point, a point representing the position of the detection unit 21, a pixel position of the image sensor 26 described later). .

The detection unit 21 includes an irradiation unit 24, an optical system 25, and an image sensor 26. The irradiation unit 24 irradiates (eg, projects) light La (eg, pattern light, irradiation light, pulsed light) onto the target area AR (space, detection area). The optical system 25 includes, for example, an imaging optical system (imaging optical system). The image sensor 26 includes, for example, a CMOS image sensor or a CCD image sensor. The image sensor 26 has a plurality of pixels arranged two-dimensionally. The image sensor 26 images the target area AR including the human body M via the optical system 25. The image sensor 26 detects light Lb (eg, reflected light, return light, transmitted light) emitted from the human body M in the target area AR by irradiation with the light La. The light Lb includes, for example, infrared light.

The detection unit 21 performs imaging based on the pattern (e.g., intensity distribution) of the light La emitted from the irradiation unit 24 and the pattern (e.g., intensity distribution, captured image) of the light Lb detected by the image sensor 26. The depth from a point on the target area AR corresponding to each pixel of the element 26 (in this case, a point on the human body M) to each pixel of the image sensor 26 is detected. As the detection result, the detection unit 21 converts a depth map (e.g., depth image, depth information, distance information) representing the depth distribution in the target area AR (e.g., human body M) into the reference point cloud data generation unit 22 ( (see Figure 1).

Note that the detection unit 21 may be a device that detects depth using a TOF (time of flight) method. The detection unit 21 may be a device that detects depth using a method other than the TOF method. The detection unit 21 may include, for example, a laser scanner (eg, a laser range finder), and may detect the depth by laser scanning. The detection unit 21 may include, for example, a phase difference sensor, and may detect the depth using a phase difference method. The detection unit 21 may detect the depth using a DFD (depth from defocus) method, for example.

Note that the detection unit 21 irradiates the human body M with infrared light or light other than infrared light (e.g., visible light), and detects the light emitted from the human body M (light reflected by the human body M, e.g., visible light). May be detected. The detection unit 21 may use energy waves such as sound waves (eg, ultrasonic waves) instead of the light La irradiated from the irradiation unit 24 . The detection unit 21 may include, for example, a stereo camera, and may detect (eg, image) the human body M from a plurality of viewpoints. The detection unit 21 may detect the depth by triangulation using captured images of the human body M from a plurality of viewpoints. The detection unit 21 may detect the depth using a method other than an optical method (eg, ultrasonic scanning).

Returning to FIG. 1, the detection result (eg, depth map) of the detection unit 21 may or may not be stored in the storage unit 23. The reference point group data generation unit 22 generates reference point group data PF as position information of the human body M based on the depth detected by the detection unit 21. The reference point group data generation unit 22 executes point group processing to generate reference point group data PF as position information of the human body M.

The reference point group data PF includes three-dimensional coordinates of a plurality of points on the human body M in the target region AR. The reference point group data PF may include three-dimensional coordinates of a plurality of points on objects around the human body M (eg, walls, floors). The reference point group data generation unit 22 calculates reference point group data PF regarding the human body M based on the position information (eg, depth) detected by the detection unit 21. The reference point group data generation section 22 reads out the detection results of the detection section 21 stored in the storage section 23 and calculates the reference point group data PF. Note that the reference point group data generation unit 22 may generate the reference point group data PF as model information (eg, shape information) of the human body M.

The detection unit 21 outputs, for example, a depth map (eg, a depth image) corresponding to the detection result. This depth map is information (eg, an image) representing the spatial distribution of depth measurements by the detection unit 21. The depth map may be a grayscale image that represents the depth at each point in the target area AR using gradation values. When the depth map is a grayscale image, parts with relatively high gradation values (e.g. white parts, bright parts) are parts with relatively small depth (relatively close to position detection unit 2). part). In the depth map, parts with relatively low gradation values (eg, black parts, dark parts) are parts with relatively large depth (parts relatively far from the position detection unit 2). Note that the depth map may be, for example, an RGB image.

Here, the reference point group data generation section 22 receives the depth map data from the detection section 21 or reads the depth map data from the storage section 23, and calculates the reference point group data PF. If the storage unit 23 is not provided, the reference point group data generation unit 22 calculates the reference point group data PF at the timing of receiving the depth map data from the detection unit 21 (for example, in real time).

For example, the reference point group data generation unit 22 calculates the three-dimensional coordinates of a point in real space corresponding to each pixel based on the gradation value (depth measurement value) of each pixel of the depth map. In the following description, the three-dimensional coordinates of one point will be referred to as point data as appropriate. The reference point group data PF is data that is a set of a plurality of point data. The reference point cloud data generation unit 22 extracts the reference point cloud data PF of the human body M by performing segmentation, pattern recognition, etc. on the entire point cloud data of the target area AR, for example.

Further, the reference point group data PF includes, for example, information representing the contour (eg, silhouette) of at least a portion of the human body M. The reference point group data PF includes, for example, information representing the extension or bending of each joint of the human body M. The reference point group data PF may include information that allows the positional relationship of each part of the human body M to be specified. For example, the reference point group data PF may include position information (eg, absolute position, relative position, coordinates) of each part of the human body M. The positional information of each part of the human body M may include, for example, the position of the end portion of the human body M (e.g., fingers, toes, head), or the position of the joints of the human body M (e.g., shoulders, elbows, wrists, knees). Or it may be both. The position information of each part of the human body M includes, for example, the position of the part between the end part of the human body M and the joint (e.g. neck, fingers), and the part between the joints of the human body M (e.g. the forearm). , upper arm, thigh (upper limb), lower leg (lower limb)).

The vertical direction (Y direction) and two orthogonal directions (X direction, Z direction) in the horizontal plane in the target area AR are given in advance to the reference point group data generation unit 22 as known information, for example. In the reference point group data PF generated by the reference point group data generation unit 22, the X, Y, and Z directions are set based on the above-mentioned known information. Note that when the human body M is moving, the moving direction of the human body M may be predetermined in a predetermined direction (e.g., the X direction) with respect to the target area AR (e.g., the field of view of the position detection unit 2). good.

In FIG. 1, the position detection section 2 is provided outside the processing device 1A. The processing device 1A is communicably connected to the position detection unit 2 by wireless or wire. The position detection section 2 outputs the reference point group data PF generated by the reference point group data generation section 22 to the processing device 1A. The processing device 1A receives (receives) the reference point group data PF output from the position detection unit 2 and processes it.

Note that the processing device 1A may acquire the reference point group data PF from the position detection unit 2 without using communication. For example, the processing device 1A may acquire the reference point group data PF of the position detection unit 2 via a storage medium such as a nonvolatile memory. For example, the storage unit 23 of the position detection unit 2 may be a card-type auxiliary storage device that is detachable from the position detection unit 2 and portable, or a storage medium such as a USB memory. The storage unit 15 removed from the processing device 1A is connected to the position detection unit 2 to import the reference point group data PF into the storage unit 15, and the storage unit 15 is returned to the processing device 1A to obtain the reference point group data PF. You may.

FIG. 3 is a functional block diagram showing an example of the processing device 1A. In addition to the generation unit 11, first generation unit 12, second generation unit 13, calculation unit 14, and storage unit 15 shown in FIG. 1, the processing device 1A includes a first setting unit 111, a feature extraction unit 112, and , a second setting section 113, a feature value calculation section 114, a determination section 115, and a resetting section 116. The storage unit 15 stores original data processed by the processing device 1A and data processed by the processing device 1A (eg, data generated by the processing device 1A). The storage unit 15 stores the reference point group data PF received from the position detection unit 2. The storage unit 15 may store the depth map output from the detection unit 21 as the original data of the reference point group data PF. In this case, the reference point group data generation section 22 of the position detection section 2 may read the depth map data from the storage section 15 and calculate the reference point group data PF.

The generation unit 11 executes a first point group data generation process of extracting points within the first region R1 from the reference point group data PF stored in the storage unit 15 and generating first point group data P1. . The first setting unit 111 executes a first area setting process for setting a first area R1 for the generation unit 11 to generate the first point group data P1.

FIG. 4 shows reference point group data PF of the human body M, and is a diagram in which a first region R1 is set for the reference point group data PF. As shown in FIG. 4, the first region R1 is a three-dimensional space. For example, the first setting unit 111 displays an image of the reference point group data PF on the display unit 3, and the user specifies a desired range on the input unit 4 to set the first region R1. Set. The image of the reference point group data PF displayed on the display section 3 may be configured to be able to three-dimensionally change its orientation by operating the input section 4 by the user.

In addition, the first setting unit 111 sets a preset range (for example, 300 mm in the X direction, 500 mm in the Y direction, 300 mm in the Z direction ) may be set as the first region R1. The range set around this point may be stored in the storage unit 15. Alternatively, a configuration may be adopted in which a plurality of ranges are stored in the storage unit 15 and the user specifies one of the ranges as the first area R1 using the input unit 4.

The first setting unit 111 also sets a predetermined ratio in the Y direction (e.g., vertical direction) to the height of the human body M, centering on the point specified by the user in the input unit 4 in the image of the reference point cloud data PF. (eg, 20%) may be set as the first region R1. The predetermined ratio may be set in advance as a fixed value, the user may select from a plurality of ratios stored in the storage unit 15 using the input unit 4, or the user may select the predetermined ratio using the input unit 4 may be input.

Further, the first setting unit 111 performs segmentation (area division) on this image data using the image data (original data in this case) of the human body M acquired by the position detection unit 2, so that the user can By selecting one or more of the plurality of regions using the input unit 4, a portion corresponding to the reference point group data PF may be set as the first region R1. The plurality of regions may be divided into parts constituting the human body M, or into standardized regions.

In addition, from the image data of the human body M, segmentation is performed as one region for each group that has the same or similar features (e.g., gradation of a grayscale image, color, pattern, pattern of an RGB image) in the image. Good too. For example, the plurality of regions may be stored in the storage unit 15 by extracting portions of the human body M from image data of the human body M by pattern matching or the like. Further, the plurality of regions may be displayed on the display unit 3 in a superimposed manner on the image of the reference point group data PF, and the first region R1 may be set by the user selecting one of the regions using the input unit 4.

Further, the first setting unit 111 includes the external shape features Q of the human body M by inputting characters (for example, arms, knees, thighs, etc.) of the external shape features Q of the human body M (see FIG. 4) using the input unit 4. The range may be set as the first region R1. The external shape feature Q is a part of the human body M that can be extracted from the imaging data of the human body M captured by the detection unit 21 of the position detection unit 2 or the reference point group data PF generated by the reference point group data generation unit 22. This is the external shape of M. In the human body M, the external shape features Q include, for example, the head, neck, shoulders, upper arms, forearms, elbows, palms, hips, buttocks, groins, thighs, lower legs, knees, heels, and the like. The external shape feature Q has a shape feature on the surface of the human body M, such as extending in a predetermined direction, protruding, concave, constricting, or bending. The feature extracting unit 112 may extract the external shape feature Q from the image data of the human body M acquired by the position detecting unit 2, for example, by pattern matching, or may extract the external shape feature Q from the image data of the human body M acquired by the position detecting unit 2. It may be extracted from the reference point group data PF, for example, from a change in three-dimensional direction when adjacent points are connected. The first setting unit 111 sets a predetermined range of the reference point group data PF as the first region R1 based on the external shape feature Q extracted by the feature extraction unit 112. The first setting unit 111 may determine a range including the external feature Q (for example, a large area including the external feature Q) from the imaging data of the human body M and set it as the first region R1, or may set the range including the external feature Q as the first region R1. A predetermined range in the X direction, Y direction, and Z direction (for example, 300 mm in the X direction, 500 mm in the Y direction, and 300 mm in the Z direction) may be set as the first region R1. In FIG. 4, "thigh" is extracted as the external feature Q, and a range including this "thigh" from a part of the "buttock (groin)" to the "ankle" is set as the first region R1. The predetermined range is stored in the storage unit 15 by a user or the like. The first region data RD1 regarding the first region R1 set by the first setting section 111 is stored in the storage section 15.

The generation unit 11 extracts each point existing within the first region R1 set by the first setting unit 111 to generate first point group data P1. The first point group data P1 may be stored in the storage unit 15. Note that one or both of the first setting section 111 and the feature extraction section 112 may be included in the generation section 11. Further, one or both of the first setting section 111 and the feature extraction section 112 may not be included in the processing device 1A. In this case, the generation unit 11 may automatically set one or more predetermined ranges from the reference point group data PF as the first region R1. Note that the first region R1 may be a region including a part of the human body M, or may be a region including the entire human body M. The first region R1 may be, for example, a region obtained by dividing the human body M in the vertical direction (eg, the Y direction), or may be a region obtained by dividing the human body M in the longitudinal direction (eg, the Z direction).

FIG. 5 is a diagram in which the first principal component D1 is calculated from the first point group data P1 of the first region R1 described above, and the second region R2 is set using the first principal component D1. The calculation unit 14 executes a first principal component calculation process of calculating the direction of the first principal component D1 by principal component analysis of the first point cloud data P1 generated by the generation unit 11. Principal component analysis is, for example, a method of multivariate analysis that synthesizes a small number of uncorrelated variables called principal components that best represent overall variation from a large number of correlated variables. In this embodiment, the axis is taken in the direction in which the variance of the first point group data P1 in the first region R1 is the largest, and this is set as the first principal component D1.

The first principal component D1 is an example of a characteristic factor calculated by multivariate analysis. The calculation unit 14 extracts a feature factor (e.g., first principal component D1) from a plurality of factors based on the first point cloud data P1 by multivariate analysis, and extracts a vector direction (e.g., first principal component D1) in this feature factor from a plurality of factors based on the first point cloud data P1. D1 direction) is calculated. The calculation unit 14 may extract a characteristic factor from a plurality of factors (e.g., variables) based on the first point cloud data P1 by multivariate analysis other than principal component analysis, and calculate the direction of a vector in this characteristic factor. . In this embodiment, a process of calculating the direction of the first principal component D1 by principal component analysis as a multivariate analysis will be described as an example.

The first generation unit 12 generates a plane RS2A (for example, a plane, see FIG. 5) that intersects the direction of the first principal component D1 calculated by the calculation unit 14 from the first point group data P1 stored in the storage unit 15. Each point in one or more second regions R2 (R2A, R2B, . . . , R2M) having Execute the process that generates the . For example, the plane RS2A is orthogonal to the direction of the first principal component D1. The second region R2 having a plane (eg, a surface) is a region (eg, a space) having a thickness in the direction of the first principal component D1. The second region R2 is, for example, a substantially plane, a rectangular parallelepiped, a cylinder, an elliptical cylinder, a polyhedron, or the like.

Note that the plane RS2A is not limited to being used, and any plane can be set as long as it intersects the direction of the first principal component D1. For example, a plane that is not orthogonal to the direction of the first principal component D1, Or it may be a curved surface. The plane RS2A is set to a size that covers a group of points near the plane RS2A when viewed from the direction of the first principal component D1.

The second setting unit 113 executes a second area setting process for setting a second area R2 for the human body M for the first generation unit 12 to generate the second point cloud data P2. The second region R2 is a three-dimensional space. One second region R2 is smaller than the first region R1. However, the second region R2 may be the same as the first region R1. The second setting unit 113 sets the length (thickness in this case) of the second region R2 in the direction of the first principal component D1. The second region R2 is a three-dimensional space defined by the length in the direction of the first principal component D1 set by the second setting unit 113 from the plane RS2A.

The second setting unit 113 sets a new second region R2B at a position different from the previously set second region R2A in the direction of the first principal component D1, and sets one end (one end side) of the first principal component D1. From there, new second regions R2B, . . . , R2M are sequentially set toward the other end (other end side). That is, the second setting unit 113 sets a plurality of second regions R2A, R2B, . . . , R2M along the direction of the first principal component D1. The plurality of second regions R2A, R2B, ..., R2M are regions having planes RS2A, RS2B, ..., RS2M, respectively. The planes RS2A, RS2B, . In this way, the second setting unit 113 creates a new second region R2 (e.g., a second region R2A) at a position different from the position of the previously set second region R2 (e.g., the second region R2A) in the direction of the first principal component D1. , second region R2B).

The plurality of second regions R2A, R2B, . . . , R2M may have the same length in the direction of the first principal component D1, or may have different lengths from each other. The second setting unit 113 may set the length of the plurality of second regions R2A, etc. in the direction of the first principal component D1 to a predetermined length (for example, 10 mm), or It may be set at a predetermined ratio (for example, 5% of the length of the first principal component D1) based on the length of the first principal component D1 within. The predetermined ratio may be stored in the storage unit 15 or may be set by the user using the input unit 4. The second setting unit 113 sets the plurality of second regions R2A, R2B, . . . , R2M to be sequentially arranged along the direction of the first principal component D1. The second setting section 113 may set the plurality of second regions R2A, R2B, . good. Further, the second setting unit 113 may set the plurality of second regions R2A, R2B, . . . , R2M so as to partially overlap each other.

Further, one or all of the plurality of second regions R2 is an analysis target region. When all of the plurality of second regions R2 are the analysis target regions, the second regions R2A, R2B, ..., R2M are the first partial region, the second partial region, ..., the Mth (M is the correct (integer) is a subregion. Further, these first to Mth partial regions are first to Mth divided regions or first to Mth small regions. The second region data RD2 regarding the second region R2 set by the second setting section 113 may be stored in the storage section 15.

The first generation unit 12 extracts each point existing in the second regions R2A, R2B, . ..., P2M). The second point group data P2 may be stored in the storage unit 15. Note that the second setting section 113 may be included in the first generation section 12. Further, the second setting unit 113 may not be included in the processing device 1A. In this case, the first generation unit 12 may automatically set one or more predetermined ranges from the first point group data P1 as the second region R2.

The feature amount calculation unit 114 executes a feature amount calculation process of calculating a feature amount from each of the plurality of second point group data P2 (P2A, P2B, . . . , P2M). The feature amount calculation unit 114 calculates the shortest path as a feature amount from each second point group data P2, for example.

FIG. 6 is a diagram showing an example of calculating the shortest path for the second point group data P2A. For example, the feature calculation unit 114 maps the point group (e.g., a plurality of point data) existing in the second point group data P2A in the direction of the first principal component D1, and The point cloud data is on an orthogonal virtual plane, and the shortest path is calculated by a method of fitting a predetermined shape (predetermined shape fitting) or a method such as the method of least squares to the point cloud expanded on this virtual plane. (See Figure 7). The predetermined shape is, for example, a shape that changes continuously. This predetermined shape may be, for example, an ellipse, an ellipse, a circle, a curved shape, or a combination of these shapes.

The feature value calculation unit 114 calculates the length of the shortest path (shortest path length) calculated by the above-described method. The shortest path length calculated by the feature value calculation unit 114 may be stored in the storage unit 15. Note that the feature quantity calculation unit 114 is not limited to calculating the shortest path as a feature quantity. The feature amount calculation unit 114 may calculate, for example, an area surrounded by a point group developed on a virtual plane or a volume of a space surrounded by the second point group data P2A as a feature amount. good. Further, the feature amount calculation unit 114 can apply the following method as a modification instead of ellipse estimation (a method of fitting an ellipse shape). For example, the feature calculation unit 114 calculates a convex hull from the second point cloud data P2 (for example, P2A), and complements the points included in this convex hull with a B-spline curve to generate the second point cloud data P2. A feature amount (eg, outer circumference length) may be calculated. Here, the convex hull is the minimum convex set including each point given in the second point group data P2. The convex hull obtained from the second point group data P2 is the intersection of all convex sets including points included in the second point group data P2, or the convex hull of all convex combinations of points included in the second point group data P2. Can be defined as a set. A convex set means, for example, that for any two points included in an object (second point group data P2 in this example), any point on a line segment connecting those two points is also included in that object. means. A convex combination is, for example, a linear combination of points (eg, vectors, scalars) having non-negative coefficients such that the sum is 1. Further, a B-spline curve is, for example, a smooth curve defined from a plurality of given points and knot vectors.

Here, the feature value calculation unit 114 may calculate the position of the first principal component D1 on the virtual plane. For example, when the feature amount calculation unit 114 performs fitting processing using an ellipse shape on a point group developed on a virtual plane, the feature value calculation unit 114 calculates the center of the ellipse shape as the position of the first principal component D1. good. Further, it can be estimated that the shortest path length calculated by the feature amount calculation unit 114 corresponds to the length of the outer circumference of that part of the human body M.

Further, the amount of data of the second point group data P2 may be small. For example, when acquiring the reference point group data PF of the human body M by the position detection unit 2, if the human body M is imaged from one direction, an image of the back side that is a shadow in that direction cannot be acquired, and as a result, the reference point The group data generation unit 22 may generate reference point group data PF without the point group on the back side.

FIG. 7 is a diagram showing another example of calculating the shortest path for the second point group data P2. Although the processing device 1A (or posture analysis system SYS) shown in FIG. 1 has a configuration including two (plural) detection units 21, the number of detection units 21 may be one. In such a processing device 1A (or posture analysis system SYS), when the human body M is imaged from one detection unit 21 with the −Z direction (for example, see FIG. 1) as a viewpoint, the surface side of the human body M (for example, the +Z side ) is imaged, but the back side (eg, -Z side) of the human body M is not imaged. Therefore, the reference point group data generation unit 22 generates the reference point group data PF (in this case, there is no point group on the back side of the human body M), and based on this reference point group data PF, the second point group data P2A1 ( (see FIG. 7) is extracted, the point group on the back side remains missing. However, it is possible to assume that the main parts of the human body M (eg, head, neck, torso, legs, arms) have a substantially elliptical external shape on a plane perpendicular to the longitudinal direction. Therefore, the feature value calculation unit 114 may calculate the shortest path by performing a fitting process on the second point group data P2A1 using an ellipse shape (see FIG. 7). Further, the feature amount calculation unit 114 may calculate the center of the ellipse shape as the position of the first principal component D1A. In addition, as shown in FIG. 7, if there is no point cloud on the back side, the detection unit 21 The human body M is imaged by the reference point cloud data generating section 22 to generate point cloud data in the second direction, and the reference point cloud data generating section 22 further generates point cloud data with the first direction as a viewpoint (for example, The reference point group data PF may be generated by combining (eg, merging) the point group data (FIG. 7) and the point group data whose viewpoint is in the second direction. The reference point group data PF synthesized in this way is, for example, like the second point group data P2A shown in FIG. and has a point cloud.

Then, the determining unit 115 calculates the two features respectively calculated by the feature amount calculating unit 114 based on the second point group data P2 (e.g., P2A and P2B) of the adjacent second regions R2 (e.g., R2A and R2B). A determination process is performed to compare the amounts (eg, shortest path length) and determine whether the amount of change exceeds a predetermined value.

FIG. 8 is a graph diagram showing the relationship between the position in the direction of the first principal component D1 and the shortest path length. For example, the determination unit 115 creates a graph in which the horizontal axis is the first principal component D1 and the vertical axis is the shortest path length (see FIG. 8), and determines whether the amount of change in the feature amount exceeds a predetermined value. . This predetermined value may be stored in the storage unit 15 in advance. Instead of determining whether the amount of change exceeds a predetermined value, the determination unit 115 may determine whether the first derivative changes.

The determination unit 115 extracts points PA and PB for which it is determined that the amount of change exceeds a predetermined value or that the first derivative changes as feature points (eg, points of change) (see FIG. 8). The determination unit 115 extracts, as points PA and PB, a portion where the shortest path length changes greatly, or a portion where the shortest path length changes from increase to decrease or from decrease to increase in the direction of the first principal component D1. The points PA and PB extracted by the determination unit 115 are stored in the storage unit 15.

When the determining unit 115 determines that the amount of change exceeds a predetermined value or that the first derivative changes, the resetting unit 116 first resets one or both of the adjacent second regions R2 used in the determination. A resetting process is executed to exclude from the set first region R1 and set a new first region R11 in a range in which the amount of change does not exceed a predetermined value. Through this resetting process, the resetting unit 116 can set a new first region R11 having a narrower range than the previous first region R1.

FIG. 9 is a diagram in which the second region R21 is set based on the newly set first region R11. For example, the resetting unit 116 excludes the second area R2 outside the points PA and PB from the plurality of second areas R2, and removes the second area R2 located outside the points PA and PB extracted by the determination unit 115. The range in the direction of the first principal component D1 is set as a new first region R11 (see FIGS. 8 and 9).

Then, when the resetting unit 116 sets a new first region R11, the generation unit 11 extracts points within the first region R11 from the reference point group data PF to generate first point group data P11. The calculation unit 14 calculates the first principal component D11 from the first point group data P11 by principal component analysis (see FIG. 9). The second setting unit 113 sets one or more second regions R21 (R21A, . . . , R21E) along the direction of the first principal component D11 (see FIG. 9).

The first generation unit 12 extracts each point existing in the second regions R21A, . . . , R21E, and generates second point group data P21A, . The feature amount calculation unit 114 calculates the shortest path length, which is a feature amount, from the second point group data P21A and the like. The determination unit 115 determines whether the amount of change in the feature amount in the adjacent second region R21 exceeds a predetermined value. If the determining unit 115 determines that the predetermined value is exceeded, the resetting unit 116 newly sets the first area as described above, and the above process is repeated.

FIG. 10 is a graph diagram showing the relationship between the position in the direction of the first principal component D11 and the shortest path length for the new second region R21. Note that the first principal component D11 corresponds to the range sandwiched between PA and PB with respect to the first principal component D1 shown in FIG. If the determination unit 115 determines that the amount of change in the feature amount in the adjacent second region R21 does not exceed a predetermined value (see FIG. 10), then the second point group data P21A, ..., P21E and the first principal component The direction and position of D11 may be stored in the storage unit 15. Furthermore, in the new first region R11, it is estimated that the change in the shortest path length along the first principal component D11 is small, or that the shortest path length does not change from increasing to decreasing or from decreasing to increasing. (See Figure 10).

In addition, in the repeated processing from the generation unit 11 to the resetting unit 116 described above, the determination unit 115 may determine the amount of change in the feature amount using the same predetermined value, or may determine the amount of change in the feature amount using a different predetermined value. You may judge. For example, the determination unit 115 may make the determination using a predetermined value smaller than the previously used predetermined value. Note that one or both of the feature value calculation unit 114 and the determination unit 115 may be included in the second setting unit 113 or the first generation unit 12. Further, one or both of the feature value calculation unit 114 and the determination unit 115 may not be included in the processing device 1A. In this case, the first generation unit 12 may simply generate the second point group data P2 from one or more second regions R2. Further, the resetting unit 116 may be included in the first setting unit 111 or the generating unit 11. If the determination unit 115 is not included in the processing device 1A, the resetting unit 116 may not be included in the processing device 1A.

The second generation unit 13 executes generation processing to generate one or both of a three-dimensional model of the human body M and posture information of the human body M, based on the second point group data P2 generated by the first generation unit 12. The second generation unit 13 includes a posture estimation unit 117, a model generation unit 118, and a rendering processing unit 119, as shown in FIG. The posture estimation unit 117 executes posture estimation processing to estimate the posture of the human body M based on the second point group data P2 generated by the first generation unit 12. The posture estimation unit 117 generates, for example, position information of a target part (eg, analysis target part, target part) in the second region R2 (R2A, etc.) of the human body M.

The posture estimation unit 117 performs recognition processing (eg, pattern recognition, shape recognition, skeleton recognition) on the shape of the target part (target part) of the human body M obtained from the second point group data P2, for example. The posture estimation unit 117 generates the position information of the target portion through recognition processing. The position information of the target portion includes, for example, the coordinates (eg, three-dimensional coordinates) of a point representing the target portion. The posture estimation unit 117 calculates the coordinates of a point representing the target portion through the above recognition process. The posture estimation unit 117 causes the storage unit 15 to store the coordinates of a point representing the target portion. Since the posture estimation unit 117 uses the second point group data P2 after performing principal component analysis, it is possible to estimate the target portion with high accuracy.

The posture estimation unit 117 obtains the position of the target part, information on the target part (e.g., attribute information, name (e.g., thigh, upper arm)), and two target parts (e.g., thigh and lower leg) as a result of the recognition process. , the lower leg and the part from the ankle to the tip of the foot, and the connection information indicating the connection relationship between the upper arm and the forearm) to generate posture information (eg, skeletal information, skeleton data). Posture estimating section 117 causes storage section 15 to store the above posture information.

The posture estimation unit 117 estimates the posture of the human body M based on the posture information. The posture estimation unit 117 determines the relative position (e.g., angle) between a first line connecting a pair of target parts that have a connection relationship and a second line that is connected to the first line and connects a pair of feature parts that have a connection relationship. Based on this, the pose of the part including the first line and the second line is estimated. The posture estimation unit 117 estimates the posture of a part or the entire human body M with high accuracy by estimating the posture of the target portion as described above.

Note that the posture estimation unit 117 may estimate the posture by referring to posture definition information that defines the posture of the human body M or the like. The above-mentioned posture definition information is, for example, a set of information indicating the type of posture and information defining the relative position of each target portion. The information representing the type of posture is, for example, the name of the posture, such as standing posture or sitting posture. The posture estimation unit 117 may estimate (identify) the posture of the human body M by comparing the posture information with the posture definition information.

In addition, the posture estimating unit 117 calculates the connected portion (human body M (joint positions) may be estimated. A plurality of target regions in an object are parts connected to each other across joints (eg, head and neck, shoulder and upper arm, upper arm and forearm, buttocks and thigh, thigh and lower leg).

FIGS. 11A and 11B are diagrams illustrating an example of estimating the posture of an object (human body M in this embodiment) by the second generation unit 13 (e.g., posture estimation unit 117) according to the present embodiment. . Referring to FIG. 11, an example in which the posture estimation unit 117 estimates joint positions in the human body M will be described using a "knee" as an example of the joint K. The posture estimation unit 117 calculates the first principal component when setting the second region R21 with respect to the first region Q1 (in this case, the thigh) that is a part (e.g., target region) of the object (human body M in this embodiment). Based on D11 and the first principal component D111 when setting the second region R121 regarding a second region Q2 (in this case, the lower leg) that is another part of the human body M and is different from the first region Q1, Information regarding the joint K, which is the connecting part between the first part Q1 and the second part Q2, is generated. After the posture (e.g., position, direction, length) of the human body M is estimated with high accuracy using the "thigh" of the human body M as the target part (first part Q1) as in the above embodiment, the technique of this embodiment is used to The first region is set to include the "lower leg" which is the target region (second region Q2) as an external feature. This first region is a part of the first region R1 (or a new first region R11) when the "thigh" is targeted, or a part of the first region R1 excluding the first region R11. Or it may include all of them. Further, the first region including the "lower leg" may be, for example, the first region R1 shown in FIG. 4. Then, once the first region is set, the processes of calculating the first principal component, setting the second region, generating the second point cloud data, and calculating the feature amount are repeated, as in the above-described embodiment. After that, as shown in FIG. 11A, the first principal component D111 is calculated from the first region R111 with the "lower leg" as the target region (second region Q2), and the first principal component D111 generated from the second region R121 is calculated. By using the two-point group data, the posture estimation unit 117 estimates the posture (eg, position, direction, length) of the "lower leg" which is the second part Q2 with high accuracy.

Subsequently, as shown in FIG. 11(B), the posture estimation unit 117 calculates the first principal component D11 when the second region R21 is set with the "thigh" (first region Q1) as the target region, and " The first principal component D111 when the second region R121 is set with the lower leg (second region Q2) as the target region is virtually extended. The first principal component D11 and the first principal component D111 may not intersect with each other and may be along planes that are parallel and spaced apart from each other, or may be in a twisted positional relationship. Then, the posture estimation unit 117 calculates the line segment LX so that the distance between the first principal component D11 and the first principal component D111 is the shortest, and defines the midpoint between the "thigh" (first part Q1) and " It is estimated that the joint K (knee) is the connecting part with the lower leg (second part Q2). Note that when the first principal component D11 and the first principal component D111 are extended and they intersect, the posture estimation unit 117 estimates the intersection as the joint K (knee). Note that the posture estimation unit 117 is not limited to estimating the midpoint of the line segment LX as the joint K. The posture estimation unit 117 may estimate a point other than the midpoint of the line segment LX as the joint K. For example, the posture estimation unit 117 may estimate a point other than the midpoint of the line segment LX to be the joint K. In addition, a point obtained by dividing the line segment LX at a predetermined ratio (for example, 3:7) may be estimated as the joint K, or a point on the first principal component D11 or a point on the first principal component D111 may be estimated. It's okay.

Further, the model generation unit 118 executes a model generation process to generate model information of the human body M based on the posture information. The model information is, for example, three-dimensional CG model data, and includes shape information of the human body M. The model generation unit 118 calculates model information of the human body M based on the second point cloud data P2 generated by the first generation unit 12. The model generation unit 118 executes surface processing to calculate surface information as shape information. The surface information includes, for example, at least one of polygon data, vector data, and draw data. The surface information includes coordinates of multiple points on the surface of the human body M and connection information between the multiple points. The connection information (eg, attribute information) includes, for example, information that associates points at both ends of a line corresponding to a ridgeline (eg, an edge) on the surface of the human body M with each other. Further, the connection information includes, for example, information that associates a plurality of lines corresponding to the outline of the surface of the human body M with each other.

In the surface processing, the model generation unit 118 estimates a surface between a point selected from a plurality of points included in the second point group data P2 and points in the vicinity thereof. In surface processing, the model generation unit 118 converts the second point group data P2 into polygon data having plane information between points. The model generation unit 118 converts the point cloud data into polygon data using, for example, an algorithm using the least squares method. The model generation unit 118 causes the storage unit 15 to store the calculated surface information.

Note that the model information may include texture information of the human body M. The model generation unit 118 may generate texture information of a surface defined by three-dimensional point coordinates and related information. The texture information includes, for example, at least one of information defining characters, figures, patterns, textures, patterns, and unevenness on the surface of the object, a specific image, and color (eg, chromatic color, achromatic color). The model generation unit 118 may cause the storage unit 15 to store the generated texture information.

The model information may also include spatial information (eg, lighting conditions, light source information) of the image. The light source information includes the position of a light source that irradiates light (e.g., illumination light) to the human body M, the direction in which the light is irradiated from this light source to the human body M (irradiation direction), the wavelength of the light irradiated from this light source, and information on at least one item of the type of light source. The model generation unit 118 may calculate the light source information using, for example, a model assuming Lambertian reflection, a model including albedo estimation, or the like. The model generation unit 118 may cause the storage unit 15 to store the generated spatial information. The model generation unit 118 does not need to generate one or both of texture information and spatial information.

The rendering processing unit 119 includes, for example, a graphics processing unit (GPU). Note that the rendering processing unit 119 may have a mode in which the CPU and memory execute each process according to an image processing program. The rendering processing unit 119 performs, for example, at least one of drawing processing, texture mapping processing, and shading processing.

In the drawing process, the rendering processing unit 119 can calculate, for example, an estimated image (eg, a reconstructed image) of the shape defined in the shape information of the generated model information viewed from an arbitrary viewpoint. The rendering processing unit 119 can reconstruct a model shape (eg, estimated image) from model information (eg, shape information) by, for example, drawing processing. The rendering processing unit 119 may cause the storage unit 15 to store the calculated estimated image data.

In the texture mapping process, the rendering processing unit 119 can calculate, for example, an estimated image in which an image indicated by the texture information of the model information is pasted onto the surface of the object on the estimated image. The rendering processing unit 119 can calculate an estimated image in which a texture different from that of the target object is pasted on the surface of the object on the estimated image. In the shading process, the rendering processing unit 119 can calculate, for example, an estimated image in which a shadow formed by a light source indicated by the light source information of the model information is added to an object on the estimated image. In the shading process, the rendering processing unit 119 can calculate, for example, an estimated image in which a shadow formed by an arbitrary light source is added to an object on the estimated image.

Note that the second generation unit 13 does not need to include the posture estimation unit 117, the model generation unit 118, and the rendering processing unit 119. Further, at least one of the posture estimation section 117, the model generation section 118, and the rendering processing section 119 may be provided in the processing device 1A outside the second generation section 13. Further, at least one of the posture estimation section 117, the model generation section 118, and the rendering processing section 119 may be provided outside the processing device 1A and communicably connected to the processing device 1A.

The processing device 1A (or posture analysis system SYS) generates information about the human body M displayed on the display section 3 by the user operating the input section 4 based on the information about the human body M generated by the second generation section 13 described above. It is possible to switch the viewpoint of a three-dimensional model or posture information. Further, the processing device 1A (or the posture analysis system SYS) can perform body shape measurement based on the information of the human body M generated by the second generation unit 13, and can also score the body shape based on the measurement result. For example, the processing device 1A displays on the display unit 3 a score representing the measurement result of the body shape of the human body M, which is measured based on the information of the human body M (e.g., posture information including a point cloud), as an absolute or relative numerical value. It is possible to display it. Note that this score is displayed on the display unit 3 using a table format or a time series graph format (eg, the horizontal axis is time or date). Note that the processing device 1A causes the display unit 3 to display the result of comparing the score based on the information on the user's human body M and the score based on the information on the teacher human body M stored in the storage unit 15 in advance. Alternatively, the result of comparing the user's past score stored in the storage unit 15 with the currently measured score may be displayed on the display unit 3. Note that the information on the user's human body M and the information on the teacher's human body M may be displayed on the display unit 3 in an overlapping manner. There is a plurality of pieces of information on the human body M serving as the teacher so that the teacher's information based on the user's proficiency level can be used, and this information is stored in the storage unit 15. The processing device 1A can change information about the human body M serving as a teacher based on the user's proficiency level. Further, the outer circumferential dimension (eg, length around the thigh) of each part of the human body M (eg, thigh, upper arm) can be displayed on the display unit 3. Note that the shortest path length calculated by the feature amount calculation unit 114 may be used as the outer circumference dimension of each part. When displaying the dimensions of each part of the human body M, it may be displayed in chronological order, or a three-dimensional model of a part or the entire human body M may be displayed on the display unit 3 in chronological order.

Next, a processing method according to this embodiment will be explained. FIG. 12 is a flowchart illustrating an example of the processing method according to this embodiment. When explaining the flowchart of FIG. 12, the above-mentioned FIGS. 1 to 10 and the like will be used as appropriate. The processing device 1A (or the posture analysis system SYS) acquires reference point group data PF of an object (in this case, the human body M) (step S1). In step S1, the reference point group data PF may be sent together from the position detection unit 2 as one file to the processing device 1A, or may be divided into multiple files and sent sequentially to the processing device 1A. .

Subsequently, the first setting unit 111 of the processing device 1A sets a first region R1 from the reference point group data PF, and the generating unit 11 extracts points within the first region R1 and generates the first point group data P1. (Step S2). In step S2, an image regarding the size and position of the set first region R1 may be displayed on the display unit 3 in a state overlaid on the reference point group data PF. Further, in step S2, the first point group data P1 may be displayed on the display unit 3 in a state separated from the reference point group data PF, or the first point group data P1 of the reference point group data PF may be displayed on the display unit 3. The display unit 3 may display the first point group data P1 in a different color from other parts or in a blinking state. Note that, as described above, the first region R1 is roughly cut out from the reference point group data PF. The first region R1 is set, for example, as a wide region that includes a region that the user desires to analyze.

Subsequently, the calculation unit 14 calculates the direction of the first principal component D1 with respect to the first point group data P1 by principal component analysis (step S3). In step S3, the calculation unit 14 may calculate the position of the first principal component D1 viewed from the direction of the first principal component D1 and the length of the first principal component D1. Then, the second setting unit 113 sets a second region R2 (R2A) having a plane (eg, plane RS2A) orthogonal to the first principal component D1 (step S4). In step S4, an image regarding the size and position of the set second region R2 may be displayed on the display unit 3 in a state overlaid on the first point group data P1 or the reference point group data PF.

Subsequently, the first generation unit 12 extracts points within the second region R2 from the first point group data P1 to generate second point group data P2 (step S5). In step S5, the second point group data P2 may be displayed on the display unit 3 in a state separated from the first point group data P1 or the reference point group data PF, or the first point group data P1 or the reference point group data The display unit 3 may display the part of the second point group data P2 in the PF in a different color from the other parts, or in a state where the part of the second point group data P2 is blinked.

Subsequently, the second generation unit 13 generates information about the human body M from the second point group data P2 (step S6). In step S5, the second generation unit 13 may display one or both of the three-dimensional model of the human body M and the posture information on the display unit 3 as information about the human body M.

The three-dimensional model and posture information (e.g., skeletal information, skeleton data) may be switchable so that either one is displayed on the display unit 3 by the user's operation of the input unit 4, or the three-dimensional model and posture information may be switchable so as to be displayed on the display unit 3. The posture information may be displayed side by side on the display unit 3. Further, for each part of the human body M, numerical values such as external dimensions may be displayed on the display unit 3 together with the three-dimensional model, and changes over time for specific external dimensions may also be displayed on the display unit 3. It may be displayed.

13 and 14 are flowcharts showing other examples of the processing method according to this embodiment. In this flowchart, steps S1 to S5 are almost the same as the processing shown in FIG. 12 described above, so the explanation will be omitted or simplified. Note that in step S4 of the flowchart in FIG. 13, the second setting unit 113 sets a plurality of second regions R2. For example, in step S4, the second setting unit 113 sequentially sets a plurality of second regions R2 along the direction of the first principal component D1.

The second setting unit 113 sets the length (in this case, the thickness) of the second region R2 in the direction of the first principal component D1, but if this length is short (in this case, if the thickness is thin) Since the number of second regions R2 increases, the subsequent processing load increases in terms of data amount. Therefore, the second setting unit 113 may set the number of second regions R2 or the length in the direction of the first principal component D1 based on the processing capacity of the processing device 1A.

Following step S5, the feature amount calculation unit 114 calculates the shortest path length as a feature amount from the second point group data P2 (step S11). Next, the first generation unit 12 or the feature amount calculation unit 114 determines whether the second region R2 of the second point group data P2 for which the shortest path length has been calculated is the last second region R2 (step S12). . If it is not the last second region R2 (NO in step S12), the process returns to step S4, and the second setting unit 113 sets the next second region R2 for the previously set second region R2. If it is the last second region R2 (YES in step S12), the determination unit 115 compares the shortest path lengths in adjacent second regions R2 and extracts the amount of change (step S13).

Subsequently, the determining unit 115 determines whether the amount of change in the shortest path length exceeds a predetermined value (step S14). If the amount of change exceeds the predetermined value (NO in step S14), the resetting unit 116 excludes one or both of the adjacent second regions R2 used for the determination from the previously set first region R1, A range in which the amount of change does not exceed a predetermined value is set as a new first region (step S15). Then, the processes from step S2 to step S14 are repeated. Through this series of processing, parts where the amount of change in the shortest path length is large are excluded, and it becomes possible to estimate the shape or posture of the part (analysis target area) desired by the user with high robustness and high accuracy.

Furthermore, if the amount of change does not exceed the predetermined value (YES in step S14), the second generation unit 13 generates information on the human body M from the second point group data P2 (step S16). This step S16 is similar to step S6 shown in FIG. 12, so the explanation will be omitted.

Note that in the flowcharts shown in FIGS. 13 and 14, in the processing from step S5 to step S12, the plurality of second regions R2 are sequentially set to calculate the shortest path length, but the process is not limited to this. For example, in step S5, the second setting unit 113 sets a plurality of second regions R2, and in step S6, the first generation unit 12 extracts a point group (point data) from each of the plurality of second regions R2. to generate a plurality of second point cloud data P2, and in step S11, the feature value calculation unit 114 calculates the shortest path length from each of the plurality of second point cloud data P2. It may be performed in parallel for a plurality of second point group data P2.

[Second embodiment]
Next, a second embodiment will be described. FIG. 15 is a diagram illustrating an example of a processing device according to the second embodiment. In this embodiment, the same components as those in the above-described embodiment are given the same reference numerals, and the description thereof will be omitted or simplified. In the processing device 1B shown in FIG. 15, a position detection unit 2 (eg, a point cloud acquisition unit) is provided (built-in) inside the processing device 1B. This processing device 1B includes a function of acquiring reference point group data PF in one unit (package). For example, a notebook computer is used as the processing device 1B, and a camera installed in the notebook computer is used as the detection unit 21.

The processing device 1B does not require a separate position detection unit 2, and is convenient when being carried. Note that in the processing device 1B, the storage unit 23 of the position detection unit 2 may also be used as the storage unit 15 included in the processing device 1B. That is, the storage unit 23 does not need to be included in the processing device 1B.

[Third embodiment]
Next, a third embodiment will be described. FIG. 16 is a diagram illustrating an example of a processing device according to the third embodiment. In this embodiment, the same components as those in the above-described embodiment are given the same reference numerals, and the description thereof will be omitted or simplified. In the processing device 1C shown in FIG. 16, a reference point cloud data generation unit 22 and a storage unit 23 of the position detection unit 2 (eg, point cloud acquisition unit) are provided (built-in) inside the processing device 1C. The detection unit 21 is provided outside the processing device 1C. The detection unit 21 only needs to have a configuration capable of capturing an image of the human body M (object), and may be, for example, a camera or a mobile terminal (eg, a smartphone, a tablet) having an image capture function. The detection unit 21 outputs the depth map as position information of the human body M to the processing device 1C (reference point group data generation unit 22).

The processing device 1C can easily generate the reference point group data PF by acquiring an image of the human body M from the detection unit 21 by simply preparing the detection unit 21 capable of imaging an object (in this case, the human body M). A laptop computer or desktop computer without an imaging function can be used. Note that in the processing device 1C, the storage section 23 of the position detection section 2 may also be used as the storage section 15 included in the processing device 1C. That is, the storage unit 23 does not need to be included in the processing device 1C.

Next, the screen displayed on the display unit 3 and the like in this embodiment will be explained. 17 to 23 are diagrams showing examples of screens displayed on the display unit 3 in the first to third embodiments. 17 to 23 illustrate, for example, screens of a body measurement application when the above-described embodiment is applied to body measurement of a human body M. FIG. 17 is the start screen SA of the body measurement application. Icons such as "Measurement", "User Information", "History", "Configuration", and "Close" are displayed on this start screen SA, allowing the user to perform desired operations. Note that in this body measurement application, the user name is "test."

When the user selects the "Mesurement" icon on the start screen SA, the measurement screen SB is displayed as shown in FIG. 18, and if "DeviceStatus" is OK, the position detection unit 2 shows a state in which the human body M can be imaged by the detection unit 21 of No. 2. When "DeviceStatus" is OK, the image captured by the above-mentioned position detection unit 2 is displayed below the icon such as "Device1" on the measurement screen SB. Further, if the "DeviceStatus" is NG, the measurement screen SB of FIG. 18 is displayed. Note that when a plurality of position detecting sections 2 or detecting sections 21 are used, the position, model name, etc. of the position detecting section 2 may be displayed below the "Device 1", "Device 2", etc. icons. Note that the above processing device causes the display unit 3 to display images captured by the respective position detection units 2 below the icons of “Device1”, “Device2”, etc., regardless of the state of “DeviceStatus”. Good too. Further, the processing device described above may display, on the measurement screen SB, a teaching image that teaches the direction in which the human body M should be imaged by the position detection unit 2 or the like.

On the measurement screen SB, an image of the posture required of the human body M at the time of imaging is displayed below the "PoseModel" icon. By considering the image of this posture, the user can easily understand the posture at the time of imaging. Furthermore, the user can change the posture displayed on the measurement screen SB by operating the "PoseModel" icon or the posture image. When changing, icons such as "PoseModel1", "PoseModel2", etc., and images of postures associated with the icons are displayed.

In FIG. 18, when the user selects the "Capture1" icon on the measurement screen SB, for example, the position detection unit 2 or the detection unit 21 shown in "Device1" images the human body M, and a reference point group is generated from the imaged data. The data generation unit 22 generates reference point group data PF. The generated reference point group data PF may be displayed below the "Device 1" icon on the measurement screen SB.

Further, when the user selects the "User Information" icon on the start screen SA, a user management screen SC is displayed as shown in FIG. 19, and user information can be registered, edited, and deleted. For example, when registering user information, a new user registration screen SC1 window as shown in FIG. 20 can be displayed. A new user can manage various data of this body measurement application by inputting predetermined items into the window of the new user registration screen SC1.

When the user selects the "History" icon on the start screen SA, as shown in FIG. 21, a data history screen SD including a list of past measurement results is displayed. On the data history screen SD, the outer circumferential dimensions of the parts of the human body M of the user are displayed in association with the date of measurement. In the data history screen SD shown in FIG. 21, the outer circumferential dimensions of "LeftUpperArm" and "Waist" are displayed in association with the date of measurement. The outer circumferential dimensions of "LeftUpperArm" and "Waist" are the second point group data when the second region R2 is "LeftUpperArm" (left upper arm) and "Waist" (waist, torso) in the description of the above embodiment. It is obtained from the three-dimensional model generated from P2. That is, in this embodiment, since a three-dimensional model is generated from the second point group data P2 subjected to principal component analysis, the outer circumferential dimensions of "LeftUpperArm" and "Waist" are measured with high accuracy.

When the user selects the "Waist" icon on the data history screen SD, a measurement position adjustment screen SD1 is displayed as shown in FIG. 22. The measurement position adjustment screen SD1 is a screen for adjusting (changing) the numerical values displayed on the data history screen SD. On the measurement position adjustment screen SD1, a plurality of positions in the + direction (upward) and - direction (downward) from the reference position (+0) and the dimensions of the outer periphery at the positions are displayed side by side in "Position Result". In the measurement position adjustment screen SD1 shown in FIG. 22, the reference position (+0) is selected. The user can adjust (change) the numerical value displayed on the data history screen SD by selecting icons at other positions (eg, +2, -3). Note that on the measurement position adjustment screen SD1, a horizontal direction line H indicating the selected position on the image is displayed. The horizontal indication line H moves according to the position selected by the user. On the measurement position adjustment screen SD1, a vertical direction line V indicating the vertical direction at the center of the human body M is displayed.

Next, when the user selects the "Configuration" icon on the start screen SA, a camera position adjustment screen SE is displayed as shown in FIG. 23. On the camera position adjustment screen SE, various settings of the device (eg, position detection unit 2), such as setting a data storage location and adjusting the camera position, can be performed, for example.

As described above, according to the present embodiment, a three-dimensional model or posture information can be generated with high precision for part or all of an object (eg, human body M). When the object is a human body M, the three-dimensional shape of each part of the human body M is generated with high precision, so the size (outer circumference dimension) of each part of the human body M can be easily and precisely acquired. Furthermore, according to the present embodiment, since the posture information of each part of the human body M is generated with high precision, the posture of each part of the human body M (for example, the thigh) during exercise (for example, ballet, baseball, golf) is generated with high precision. orientation, upper arm orientation, ballet form, pitching form, batting form, golf swing form) can be easily and accurately acquired.

Note that the processing device 1A (or posture analysis system SYS) described above may include two or more position detection units 2. Further, the posture analysis system SYS may include two or more processing devices 1A, 1B, and 1C.

In the embodiments described above, the generation unit 11, the first generation unit 12, the second generation unit 13, the calculation unit 14, the first setting unit 111, the feature extraction unit 112, and the second setting unit 113 of the processing devices 1A, 1B, and 1C. , the feature calculation unit 114, the determination unit 115, the resetting unit 116, the posture estimation unit 117, the model generation unit 118, the rendering processing unit 119, and the reference point group data generation unit 22 of the position detection unit 2, By being read, the software and hardware resources may be realized as a concrete means of cooperation.

Furthermore, in the embodiments described above, the processing devices 1A, 1B, and 1C include, for example, computer systems. The processing devices 1A, 1B, and 1C read processing programs stored in the storage unit 15 and execute various processes according to the processing programs. This processing program, for example, causes a computer to obtain the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and to calculate the direction of the first principal component from the first point cloud data. The second point group data is generated by extracting points within a region that intersects the direction, and the object information is generated from the second point group data. This program may be provided by being recorded on a computer-readable storage medium (eg, non-transitory tangible media).

Although the embodiments have been described, the technical scope of the present invention is not limited to the content described in the above-mentioned embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. Furthermore, the requirements described in the above embodiments and the like can be combined as appropriate. Moreover, the execution order of each process shown in the embodiment can be implemented in any order as long as the output of the previous process is not used in the subsequent process. Further, even if the processing in the above-described embodiment is explained using "first", "next", "sequently", "and", etc. for convenience, it is not essential that the processing be performed in this order. In addition, to the extent permitted by law, the disclosures of all documents cited in the above-mentioned embodiments are incorporated into the description of the main text.

Note that the technical scope of the present invention is not limited to the aspects described in the above-mentioned embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. Furthermore, the requirements described in the above embodiments and the like can be combined as appropriate. In addition, to the extent permitted by law, the disclosures of Japanese Patent Application No. 2018-143688, which is a Japanese patent application, and all documents cited in the above-mentioned embodiments are incorporated into the description of the main text.

D1, D11...First principal component M...Human body (object) K...Joint (connection part) LX...Line segment P1, P11...First point group data P2, P2A to P2M・...Second point group data PF...Reference point group data Q...External feature Q1...First part Q2...Second part R1, R11...First region R2, R2A, R2B~ R2M, R21A to R21E...Second region RS2A...Plane (plane) SYS...Posture analysis system 1A, 1B, 1C...Processing device 2...Position detection unit (point cloud acquisition unit) 11 ...Generation unit 12...First generation unit 13...Second generation unit 14...Calculation unit 22...Reference point group data generation unit 111...First setting unit 112...Characteristics Extraction unit 113... Second setting unit 114... Feature value calculation unit 115... Judgment unit 116... Resetting unit

Claims (12)

an acquisition unit that acquires first point cloud data;
A principal component direction is determined by principal component analysis of the first point cloud data, and a second point is extracted from each point existing in a region intersecting the principal component direction at a plurality of positions in the principal component direction. a calculation unit that calculates the feature amount of the shape indicated by the group data ;
a determination unit that determines a change in the feature amount at a plurality of positions in the principal component direction;
A processing device comprising: The processing device according to claim 1, wherein the calculation unit calculates, as the feature amount, a path length of a shape indicated by the second point group data. The processing device according to claim 1, wherein the calculation unit calculates, as the feature amount, an area surrounded by a shape indicated by the second point group data. 4. The method according to claim 1, wherein the shape is an ellipse, and the center of the ellipse is determined as the position of the principal component obtained by principal component analysis of the first point group data. Processing equipment. The shape is between a first surface that intersects the principal component direction at a first position in the principal component direction and a second surface that intersects the principal component direction at a second position in the principal component direction. Any one of claims 2 to 4, wherein the point group has a shape shown when each of the point groups existing in is moved in the principal component direction and mapped into a plane intersecting the principal component direction. The processing device according to item 1. The calculation unit includes, as the feature amount, a first surface that intersects the principal component direction at a first position in the principal component direction, and a first surface that intersects the principal component direction at a second position in the principal component direction. The processing device according to claim 1, which calculates the volume of a region surrounded by two surfaces and a point group existing between the first surface and the second surface. a setting unit configured to set first point group first partial data, which is a portion of the first point group data, from the first point group data based on the determination of the determination unit;
A processing device according to any one of claims 1 to 6 . The calculation unit determines a first direction indicating a direction of a new principal component calculated by principal component analysis of the first point group first partial data.
The processing device according to claim 7 . Equipped with a model generation unit that generates model information from point cloud data,
The setting unit sets second partial data of the first point group different from the first partial data of the first point group,
The calculation unit determines a second direction indicating a direction of a new principal component calculated by principal component analysis of the first point group second partial data,
The processing device according to claim 8 , wherein the model generation unit generates information indicating a connection portion between the first part and the second part of the model information from the first direction and the second direction. The processing device according to claim 9 , wherein the model information includes at least one of shape information generated from the point cloud data and posture information generated from the point cloud data. A processing device according to any one of claims 1 to 10 ,
A posture analysis system comprising: a detection unit that detects distance information to an object and generates point cloud data from the distance information. A program that causes a computer to execute,
Obtain the first point cloud data,
A principal component direction is determined by principal component analysis of the first point cloud data, and a second point is extracted from each point existing in a region intersecting the principal component direction at a plurality of positions in the principal component direction. Calculate the feature amount of the shape indicated by the group data ,
A program that executes processing for determining changes in the feature amount at a plurality of positions in the principal component direction.

Images