JP7147848B2 - Processing device, posture analysis system, processing method, and processing program - Google Patents

Description

The present invention relates to a processing device, a posture analysis system, a processing method, and a processing program.

As a technique for detecting an object, for example, there is a technique described in Patent Document 1 below, but it is desired that the technique for detecting an object can detect the shape or orientation of all or part of the object with high accuracy. ing.

JP 2007-288682 A

According to the aspect of the present invention, a calculation unit for obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object; A first generating unit that generates second point cloud data by extracting points in a region that intersects with a direction; a quantity calculation unit; a resetting unit that resets the range of the first point cloud data based on a change in the direction of the first principal component of the feature quantity; a generation unit, wherein the calculation unit obtains the direction of the first principal component from the first point cloud data included in the range of the first point cloud data reset by the resetting unit.
According to the aspect of the present invention, a calculation unit for obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object; A first generation unit that generates second point cloud data by extracting points in an area that intersects with a direction; a feature amount calculation unit that calculates a feature amount from the second point cloud data; a second generation unit that generates information, and the feature amount calculation unit calculates the shortest path of the second point cloud data as the feature amount.
According to the aspect of the present invention, a calculation unit for obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object; a first generation unit that generates second point cloud data by extracting points in an area that intersects a direction; and a second generation unit that generates object information based on the second point cloud data, the second generation unit is based on the first principal component of the region set for the first portion of the object and the first principal component of the region set for the second portion different from the first portion of the object, the first portion and the second A processing device is provided that generates information about connections with sites.
According to the aspect of the present invention, a calculating unit that extracts feature factors from a plurality of factors based on the first point cloud data based on the position information of each point on the object and obtains the direction of the vector in the feature factors; A first generation unit that generates second point cloud data by extracting points in the region, and a second generation unit that generates object information based on the second point cloud data. The direction of the vector from which the feature factor is extracted is obtained, and the second generation unit generates the feature factor of the area set for the first part of the object and the feature factor of the area set for the second part different from the first part of the object. A processing device is provided for generating information about a connection between a first portion and a second portion based on and.
According to the aspect of the present invention, a calculation unit for obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and the first principal component from the first point cloud data Provided is a processing device comprising: a first generation unit that generates second point cloud data by extracting points in an area that intersects with the direction of; and a second generation unit that generates object information based on the second point cloud data be done.

According to the aspect of the present invention, a calculating unit that extracts a feature factor from a plurality of factors based on first point cloud data based on position information of each point on an object and obtains the direction of a vector in the feature factor; A processing device is provided that includes a first generation unit that generates second point cloud data by extracting points in an area that is to be measured, and a second generation unit that generates object information from the second point cloud data.

According to an aspect of the present invention, a posture comprising: the processing device according to the aspect ; An analysis system is provided.

According to the aspect of the present invention, the direction of the first principal component is obtained by principal component analysis of the first point cloud data based on the position information of each point on the object, and the direction of the first principal component from the first point cloud data. Generating second point cloud data by extracting points in a region that intersects with , calculating feature quantities at a plurality of positions in the direction of the first principal component from the second point cloud data, and calculating feature quantities resetting the range of the first point cloud data based on a change in orientation of the first principal component of the first principal component; and generating object information from the second point cloud data; In calculating the direction, a processing method is provided for obtaining the direction of the first principal component from the first point cloud data included in the reset range of the first point cloud data.
According to the aspect of the present invention, the direction of the first principal component is obtained by principal component analysis of the first point cloud data based on the position information of each point on the object, and the direction of the first principal component is obtained from the first point cloud data. A processing method is provided that includes generating second point cloud data that extracts points in a region that intersects directions, and generating object information from the second point cloud data.

According to the aspect of the present invention, the computer obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and obtains the direction of the first principal component from the first point cloud data. Generating second point cloud data by extracting points in an area intersecting with the direction of the component, and calculating feature amounts at a plurality of positions in the direction of the first principal component from the second point cloud data. , resetting the range of the first point cloud data based on a change in the direction of the first principal component of the feature quantity, and generating object information from the second point cloud data; In calculating the direction of one principal component, a processing program is provided for obtaining the direction of the first principal component from the first point cloud data included in the reset range of the first point cloud data.
According to the aspect of the present invention, the computer obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and obtains the direction of the first principal component from the first point cloud data. Generating second point cloud data by extracting points in an area intersecting with the component direction, calculating a feature amount from the second point cloud data, and generating object information from the second point cloud data A processing program is provided for executing (1) and (2), and calculating the shortest path of the second point cloud data as the feature amount in the calculation of the feature amount.
According to the aspect of the present invention, the computer obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and obtains the direction of the first principal component from the first point cloud data. Generating second point cloud data by extracting points in an area intersecting with the direction of the component, and generating object information from the second point cloud data, and generating the object information, Based on the first principal component of the region set for the first portion of the object and the first principal component of the region set for the second portion different from the first portion of the object, the first portion and the second portion are determined. A processing program is provided for generating information about the connecting portion of the.
According to the aspect of the present invention, the computer extracts feature factors from a plurality of factors based on the first point cloud data based on the position information of each point on the object, obtains the direction of the vector in the feature factor; Generating second point cloud data by extracting points in the intersecting region and generating object information from the second point cloud data are executed, and in calculating the direction of the vector, principal component analysis The direction of the vector from which the feature factor is extracted is obtained, and in the generation of the object information, the feature factor of the area set for the first part of the object and the feature of the area set for the second part different from the first part of the object are obtained. A processing program is provided that generates information about a connection between a first portion and a second portion based on a factor.
According to the aspect of the present invention, the computer obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and obtains the direction of the first principal component from the first point cloud data. A processing program is provided for generating second point cloud data by extracting points in an area intersecting the direction of a principal component, and generating object information from the second point cloud data.

It is a figure which shows an example of the processing apparatus (posture analysis system) which concerns on 1st Embodiment. It is a figure which shows an example of the position detection part (point group acquisition part) which concerns on this embodiment. 2 is a functional block diagram showing an example of a processing device shown in FIG. 1; FIG. FIG. 4 is a diagram in which a first region is set for reference point cloud data according to the embodiment; FIG. 4 is a diagram in which a first principal component is calculated from first point cloud data of a first region and a second region is set according to the present embodiment; It is a figure which shows an example which calculated the shortest path|route about the 2nd point cloud data which concerns on this embodiment. It is a figure which shows the other example which calculated the shortest path|route about the 2nd point cloud data which concerns on this embodiment. FIG. 5 is a graph showing the relationship between the position in the direction of the first principal component and the shortest path length according to the embodiment; It is the figure which set the 2nd area|region from the new 1st area|region which concerns on this embodiment. FIG. 11 is a graph showing the relationship between the position in the direction of the first principal component and the shortest path length for the new second region according to the present embodiment; (A) and (B) are diagrams showing an example of estimating the orientation of an object by a second generation unit according to the embodiment. It is a flow chart which shows an example of the processing method concerning this embodiment. 6 is a flow chart showing another example of the processing method according to the present embodiment; FIG. 14 is a flow chart showing another example of the processing method following FIG. 13; FIG. It is a figure which shows an example of the processing apparatus which concerns on 2nd Embodiment. It is a figure which shows an example of the processing apparatus which concerns on 3rd Embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment. It is a figure which shows an example of the screen displayed on the display part which concerns on this embodiment.

Hereinafter, embodiments will be described with reference to the drawings. In the drawings, in order to describe the embodiments, the scale is appropriately changed, such as by enlarging or emphasizing a portion. In the drawings, there are diagrams for explaining the directions in the drawings using the XYZ coordinate system. In this XYZ coordinate system, the plane parallel to the horizontal plane is the XZ plane. In this XZ plane, the direction of the shoulder width of the human body M is expressed as the X direction, and the direction orthogonal to the X direction is expressed as the Z direction. A direction perpendicular to the XZ plane is referred to as a Y direction. Regarding each of the X direction, Y direction, and Z direction, the direction indicated by the arrow in the drawing is the + direction, and the direction opposite to the direction indicated by the arrow is the − direction.

[First embodiment]
A first embodiment will be described. FIG. 1 is a diagram showing an example of a processing apparatus 1A according to the first embodiment. FIG. 1 also shows an example of the posture analysis system SYS according to the embodiment. The processing device 1A and the posture analysis system SYS generate one or both of posture information and shape information of the object M. FIG. In the present embodiment, a human body will be described as an example of the object M, but the object M may be, for example, an animal, a humanoid robot, an animal robot other than a human, or a non-animal robot. Hereinafter, the object M will be referred to as a human body M for explanation.

The human body M assumes a predetermined stationary posture (eg, a specified pose, standing state) in a target area AR (eg, detection area, field of view of the position detection section 2) to be detected by the position detection section 2, which will be described later. or in a state of movement (eg, change, action). The human body M changes, for example, one or both of its posture and shape as it moves. The moving human body M is used in sports such as ballet, fencing, baseball, soccer, golf, kendo, American football, ice hockey, or gymnastics, running, exercise, yoga, bodybuilding, walking or posing such as fashion shows, and games. , training, dance, traditional performing arts, person recognition, or working human body. The human body M may include other objects attached to the human body M (eg, clothing, wearables, exercise equipment, armor).

The processing device 1A (or posture analysis system SYS) is, for example, a shape evaluation device, a motion detection device, a motion evaluation device, a motion capture device, a motion detection system, a motion support system, or the like. Posture analysis system SYS includes a processing device 1A and a position detector 2 . The position detection unit 2 is, for example, a point group acquisition unit. The posture analysis system SYS may include a plurality of processing devices 1A or a plurality of position detection units 2 (for example, point group acquisition units). A processing device 1A (or posture analysis system SYS) of this embodiment includes two position detection units 2 . Although the two position detection units 2 have the same configuration, they may have different configurations. One of the two position detection units 2 has a viewpoint set to the front side of the human body M (eg, +Z side), and the other has a viewpoint set to the back side of the human body M (eg, -Z side). The two position detectors 2 may be arranged symmetrically about the human body M, or may be arranged asymmetrically. Based on the reference point cloud data PF obtained from the surface of the human body M, the processing device 1A (or the posture analysis system SYS) generates three-dimensional data, which is information on the shape or posture of the human body M. The reference point cloud data PF is information representing the shape of the human body M (eg, three-dimensional shape).

The processing device 1A includes a generation unit 11, a first generation unit 12, a second generation unit 13, a calculation unit 14, and a storage unit 15, as shown in FIG. A part or the whole of the processing device 1A may be a portable device (eg, information terminal, smart phone, tablet, camera-equipped mobile phone, wearable terminal). Also, a part or the whole of the processing device 1A may be a stationary device (for example, a desktop personal computer).

Next, each part which comprises 1 A of processing apparatuses is demonstrated. The generation unit 11 extracts each point in the first region R1 (see FIG. 5, for example) from the reference point cloud data PF (see FIG. 4, for example) which is the positional information of each point on the human body M, and generates a first 1 point cloud data P1 is generated. The calculation unit 14 calculates the direction of the first principal component D1 (eg, see FIG. 5) in the first point cloud data P1 by subjecting the first point cloud data P1 generated by the generation unit 11 to principal component analysis. Then, the first generation unit 12 generates a second region having a plane RS2A (eg, see FIG. 5) that is a plane that intersects the direction of the first principal component D1 calculated by the calculation unit 14 from the first point cloud data P1. Points within R2 (eg, see FIG. 5) are extracted to generate second point cloud data P2. The second generator 13 generates information about the human body M from the second point cloud data P2 generated by the first generator 12 . Details of the generating unit 11, the first generating unit 12, the second generating unit 13, and the calculating unit 14 will be described later. Note that the first region R1 shown in FIG. 4 is a region smaller than the reference region formed by the reference point cloud data PF.

The storage unit 15 is, for example, a volatile or nonvolatile memory, hard disk (HDD), solid state drive (SSD), or the like. The processing device 1A (or the posture analysis system SYS) includes a display section 3 and an input section 4 . The display unit 3 displays the image based on the image data output from the processing device 1A. For example, the processing device 1</b>A outputs the data of the model image of the human body M generated by the second generation unit 13 or the estimated image of the human body M to the display unit 3 . Then, the display unit 3 displays the estimated image based on the estimated image data output from the processing device 1A. The display unit 3 includes, for example, a liquid crystal display and an organic EL display.

The input unit 4 is used to input various information (eg, data, instructions) to the processing device 1A. By operating the input unit 4, the user can input various information to the processing device 1A. The input unit 4 includes, for example, at least one of a keyboard, mouse, trackball, touchpad, and voice input device (eg, microphone). Note that the display unit 3 and the input unit 4 may be touch panels. The processing device 1A (or the posture analysis system SYS) does not have to include the display section 3 . For example, the processing device 1A may output data such as an image to another display unit or display device via communication, and the display unit or the like may display the image. Further, the processing device 1A (posture analysis system SYS) does not have to include the input unit 4 . For example, the processing device 1A may be configured such that various commands and information are input via communication.

The position detection unit 2 detects position information of the human body M. FIG. The position detection unit 2 detects point cloud data including the three-dimensional coordinates of each point on the surface of the human body M as the positional information of the human body M. FIG. The position detection unit 2 includes a detection unit 21, a reference point cloud data generation unit 22, and a storage unit 23, as shown in FIG. The storage unit 23 is, for example, a volatile or nonvolatile memory, hard disk (HDD), solid state drive (SSD), or the like. Note that the position detection unit 2 does not have to include the storage unit 23 .

The detection unit 21 is, for example, at least part of a portable device (portable device). The detector 21 may be at least part of a stationary device. The reference point cloud data generator 22 may be provided in a device external to the processing device 1A (for example, may be externally connected). For example, the position detection unit 2 may be a device external to the processing device 1A, and may incorporate the detection unit 21, the reference point cloud data generation unit 22, and the storage unit 23. A part or the whole of the position detection unit 2 may be a portable device (eg, information terminal, smart phone, tablet, camera-equipped mobile phone, wearable terminal). Also, part or the whole of the position detection unit 2 may be a stationary device (eg, a fixed-point camera).

FIG. 2 is a diagram showing an example of the detection unit 21 of the position detection unit 2 (eg, point cloud acquisition unit). The detection unit 21 detects the depth as the positional information of the human body M. FIG. The detection unit 21 includes a depth sensor (eg, depth camera). The detection unit 21 detects the depth (eg, position information, distance, depth, depth) from a predetermined point to each point on the surface of the human body M placed in the target area AR. The predetermined point is, for example, a point at a position that serves as a reference for detection by the detection unit 21 (eg, a point of view, a detection source point, a point representing the position of the detection unit 21, a pixel position of an imaging element 26 described later). .

The detection unit 21 includes an irradiation unit 24 , an optical system 25 and an imaging device 26 . The irradiation unit 24 irradiates (eg, projects) light La (eg, pattern light, irradiation light, pulsed light) onto a target area AR (space, detection area). The optical system 25 includes, for example, an imaging optical system (imaging optical system). The imaging device 26 includes, for example, a CMOS image sensor or a CCD image sensor. The imaging device 26 has a plurality of pixels arranged two-dimensionally. The imaging element 26 images the target area AR including the human body M via the optical system 25 . The imaging element 26 detects light Lb (eg, reflected light, returned light, transmitted light) emitted from the human body M in the target area AR by the irradiation of the light La. The light Lb includes infrared light, for example.

The detection unit 21 captures an image based on the pattern (eg, intensity distribution) of the light La emitted from the irradiation unit 24 and the pattern (eg, intensity distribution, captured image) of the light Lb detected by the imaging device 26. The depth from the point on the target area AR corresponding to each pixel of the device 26 (in this case, the point on the human body M) to each pixel of the imaging device 26 is detected. As the detection result, the detection unit 21 sends a depth map (eg, depth image, depth information, distance information) representing the depth distribution in the target area AR (eg, human body M) to, for example, the reference point cloud data generation unit 22 ( (see Fig. 1).

Note that the detector 21 may be a device that detects depth by a TOF (time of flight) method. The detector 21 may be a device that detects depth by a method other than the TOF method. The detection unit 21 may include, for example, a laser scanner (eg, a laser rangefinder) and detect the depth by laser scanning. The detection unit 21 may include, for example, a phase difference sensor and detect the depth by a phase difference method. The detection unit 21 may detect the depth by, for example, a DFD (depth from defocus) method.

The detection unit 21 irradiates the human body M with infrared light or light other than infrared light (eg, visible light), and detects light emitted from the human body M (light reflected from the human body M, eg, visible light). may be detected. The detection unit 21 may use energy waves such as sound waves (eg, ultrasonic waves) instead of the light La emitted from the irradiation unit 24 . The detection unit 21 may include, for example, a stereo camera, etc., and may detect (eg, image) the human body M from a plurality of viewpoints. The detection unit 21 may detect the depth by triangulation using captured images of the human body M captured from a plurality of viewpoints. The detection unit 21 may detect the depth by a technique other than an optical technique (eg, scanning using ultrasonic waves).

Returning to FIG. 1 , the detection result (eg, depth map) of the detection unit 21 may or may not be stored in the storage unit 23 . The reference point cloud data generation unit 22 generates reference point cloud data PF as position information of the human body M based on the depth detected by the detection unit 21 . The reference point cloud data generation unit 22 performs point cloud processing for generating reference point cloud data PF as position information of the human body M. FIG.

The reference point cloud data PF includes three-dimensional coordinates of a plurality of points on the human body M in the target area AR. The reference point cloud data PF may include three-dimensional coordinates of a plurality of points on objects around the human body M (eg walls, floors). The reference point cloud data generation unit 22 calculates reference point cloud data PF regarding the human body M based on the position information (eg, depth) detected by the detection unit 21 . The reference point cloud data generation unit 22 reads the detection result of the detection unit 21 stored in the storage unit 23 and calculates reference point cloud data PF. Note that the reference point cloud data generation unit 22 may generate the reference point cloud data PF as model information (eg, shape information) of the human body M. FIG.

The detection unit 21 outputs, for example, a depth map (eg, depth image) corresponding to the detection result. The depth map is information (eg, image) representing the spatial distribution of the depth measurement values obtained by the detection unit 21 . The depth map may be a grayscale image in which the depth at each point of the target area AR is represented by a gradation value. When the depth map is a grayscale image, portions with relatively high gradation values (e.g., white portions and bright portions) correspond to portions with relatively small depths (relatively close to the position detection unit 2). part). In the depth map, portions with relatively low gradation values (eg, black portions and dark portions) are portions with relatively large depths (portions relatively far from the position detection unit 2). Note that the depth map may be, for example, an RGB image.

Here, the reference point cloud data generation unit 22 receives the depth map data from the detection unit 21, or reads the depth map data from the storage unit 23, and calculates the reference point cloud data PF. If the storage unit 23 is not provided, the reference point cloud data generation unit 22 calculates the reference point cloud data PF at the timing of receiving the depth map data from the detection unit 21 (for example, in real time).

For example, the reference point cloud data generation unit 22 calculates the three-dimensional coordinates of the point on the real space corresponding to each pixel based on the gradation value (measured value of depth) of each pixel of the depth map. In the following description, three-dimensional coordinates of one point will be referred to as point data as appropriate. The reference point cloud data PF is a set of point data. The reference point cloud data generator 22 extracts the reference point cloud data PF of the human body M by, for example, segmenting and pattern recognition the point cloud data of the entire target area AR.

Further, the reference point cloud data PF includes, for example, information representing at least a part of the contour (eg, silhouette) of the human body M. The reference point cloud data PF includes, for example, information representing the extension or bending of each joint of the human body M. The reference point cloud data PF may include information that can identify the positional relationship of each part of the human body M. For example, the reference point cloud data PF may include position information of each part of the human body M (eg, absolute position, relative position, coordinates). The position information of each part of the human body M is, for example, one of the positions of the terminal parts of the human body M (eg, fingers, toes, head) and the positions of the joints of the human body M (eg, shoulders, elbows, wrists, knees). or both. The position information of each part of the human body M is, for example, the position of the part between the terminal part of the human body M and the joint (eg, neck, fingers), and the part between the joints of the human body M (eg, the forearm). , upper arm, thigh (upper limb), lower leg (lower limb)) or both.

The vertical direction (Y direction) in the target area AR and two orthogonal directions (X direction and Z direction) in the horizontal plane are given in advance to the reference point cloud data generator 22, for example, as known information. In the reference point cloud data PF generated by the reference point cloud data generation unit 22, each direction of X, Y, and Z is set based on the above known information. Note that when the human body M is moving, even if the moving direction of the human body M is predetermined in a predetermined direction (eg, X direction) with respect to the target area AR (eg, the field of view of the position detection unit 2), good.

In FIG. 1, the position detection unit 2 is provided outside the processing device 1A. The processing device 1A is communicably connected to the position detection unit 2 wirelessly or by wire. The position detection unit 2 outputs the reference point cloud data PF generated by the reference point cloud data generation unit 22 to the processing device 1A. The processing device 1A receives (receives) the reference point cloud data PF output by the position detection unit 2 and processes it.

Note that the processing device 1A may acquire the reference point cloud data PF from the position detection unit 2 without communication. For example, the processing device 1A may acquire the reference point cloud data PF of the position detection unit 2 via a storage medium such as a nonvolatile memory. For example, the storage unit 23 of the position detection unit 2 may be a card-type auxiliary storage device that can be removed from the position detection unit 2 and carried, or a storage medium such as a USB memory. The reference point cloud data PF is acquired by connecting the storage unit 15 removed from the processing device 1A to the position detection unit 2, loading the reference point cloud data PF into the storage unit 15, and returning the storage unit 15 to the processing device 1A. You may

FIG. 3 is a functional block diagram showing an example of the processing device 1A. In addition to the generation unit 11, the first generation unit 12, the second generation unit 13, the calculation unit 14, and the storage unit 15 shown in FIG. , a second setting unit 113 , a feature amount calculation unit 114 , a determination unit 115 , and a resetting unit 116 . The storage unit 15 stores original data processed by the processing device 1A and data processed by the processing device 1A (eg, data generated by the processing device 1A). The storage unit 15 stores the reference point cloud data PF received from the position detection unit 2 . The storage unit 15 may store the depth map output from the detection unit 21 as the original data of the reference point cloud data PF. In this case, the reference point cloud data generation unit 22 of the position detection unit 2 may read the depth map data from the storage unit 15 and calculate the reference point cloud data PF.

The generation unit 11 extracts points in the first region R1 from the reference point cloud data PF stored in the storage unit 15 and executes first point cloud data generation processing to generate first point cloud data P1. . The first setting unit 111 executes first region setting processing for setting a first region R1 for the generation unit 11 to generate the first point cloud data P1.

FIG. 4 shows the reference point cloud data PF of the human body M, and is a diagram in which the first region R1 is set for the reference point cloud data PF. As shown in FIG. 4, the first region R1 is a three-dimensional space. For example, the first setting unit 111 causes the display unit 3 to display an image of the reference point cloud data PF, and the user designates a desired range on the image using the input unit 4 to set the first region R1. set. The image of the reference point cloud data PF displayed on the display unit 3 may be changed three-dimensionally by the operation of the input unit 4 by the user.

In addition, the first setting unit 111 sets a preset range (eg, 300 mm in the X direction, 500 mm in the Y direction, 300 mm in the Z direction) around the point specified by the user in the input unit 4 in the image of the reference point cloud data PF. ) may be set as the first region R1. The range set around this point may be stored in the storage unit 15 . Alternatively, the storage unit 15 may store a plurality of ranges, and the user may specify one of the ranges as the first region R1 using the input unit 4 .

Further, the first setting unit 111 sets a predetermined ratio in the Y direction (for example, the vertical direction) with respect to the height of the human body M, centering on the point specified by the user with the input unit 4 in the image of the reference point cloud data PF. A range (eg, 20%) may be set as the first region R1. The predetermined ratio may be set in advance as a fixed value, may be selected by the user through the input unit 4 from a plurality of ratios stored in the storage unit 15, or may be selected by the user through the input unit. 4 may be entered.

In addition, the first setting unit 111 uses the image data of the human body M acquired by the position detection unit 2 (original data in this case) to perform segmentation (area division) on this image data, so that the user can A portion corresponding to the reference point cloud data PF may be set as the first region R1 by selecting one or more of the plurality of regions using the input unit 4 . The plurality of regions may be divided for each part constituting the human body M, or may be divided for each standardized region.

Further, from the image data of the human body M, segmentation is performed as one region for each group having the same or similar feature amount (eg, gray scale image gradation, RGB image color, pattern, pattern) in the image. good too. The plurality of regions may be stored in the storage unit 15 by extracting, for example, portions forming the human body M from image data of the human body M by pattern matching or the like. Alternatively, the plurality of regions may be displayed on the display unit 3 while being superimposed on the image of the reference point cloud data PF, and the first region R1 may be set by the user selecting one of the regions using the input unit 4.

In addition, the first setting unit 111 includes the external feature Q of the human body M by inputting the external feature Q (see FIG. 4) of the human body M using the input unit 4 as characters (eg, arms, knees, thighs, etc.). The range may be set as the first region R1. The external feature Q is a part of the human body M that can be extracted from the imaging data of the human body M captured by the detection unit 21 of the position detection unit 2 or the reference point cloud data PF generated by the reference point cloud data generation unit 22. It is an external part of M. The external features Q of the human body M include, for example, the head, neck, shoulders, upper arms, forearms, elbows, palms, hips, buttocks, groins, thighs, lower legs, knees, and heels. The external shape feature Q has a shape feature on the surface of the human body M, such as extending in a predetermined direction, protruding, recessed, constricted, or bent. For example, the feature extraction unit 112 may extract the external feature Q from the image data of the human body M acquired by the position detection unit 2, for example, by pattern matching or the like. From the reference point cloud data PF, for example, it may be extracted from a three-dimensional direction change when adjacent points are connected. The first setting unit 111 sets a predetermined range in the reference point cloud data PF as the first region R1 based on the outline feature Q extracted by the feature extraction unit 112 . The first setting unit 111 may determine a range including the external feature Q (for example, a large area including the external feature Q) from the imaging data of the human body M and set it as the first area R1. A predetermined range (eg, 300 mm in the X direction, 500 mm in the Y direction, and 300 mm in the Z direction) may be set as the first region R1 with respect to the center. In FIG. 4, "thighs" are extracted as the outer shape feature Q, and a range including a portion of the "buttocks (crotch)" to the "ankles" is set as the first region R1. The predetermined range is stored in the storage unit 15 by the user or the like. First region data RD1 related to first region R1 set by first setting unit 111 is stored in storage unit 15 .

The generation unit 11 extracts each point existing in the first region R1 set by the first setting unit 111 to generate the first point cloud data P1. The first point cloud data P1 may be stored in the storage unit 15 . One or both of the first setting unit 111 and the feature extraction unit 112 may be included in the generation unit 11 . Also, one or both of the first setting unit 111 and the feature extraction unit 112 may not be provided in the processing device 1A. In this case, the generator 11 may automatically set one or a plurality of predetermined ranges from the reference point cloud data PF as the first region R1. The first region R1 may be a region including part of the human body M, or may be a region including the entire human body M. The first region R1 may be, for example, a region obtained by dividing the human body M in the vertical direction (eg, Y direction), or may be a region obtained by dividing the human body M in the front-rear direction (eg, Z direction).

FIG. 5 is a diagram in which the first principal component D1 is calculated from the first point cloud data P1 of the first region R1, and the second region R2 is set using the first principal component D1. The calculation unit 14 performs a first principal component calculation process of calculating the direction of the first principal component D1 by subjecting the first point cloud data P1 generated by the generation unit 11 to principal component analysis. Principal component analysis is, for example, a method of multivariate analysis that synthesizes a small number of uncorrelated variables called principal components that best represent the overall variation from a large number of correlated variables. In the present embodiment, the direction in which the variance of the first point cloud data P1 in the first region R1 is greatest is taken as the axis, and this is taken as the first principal component D1.

The first principal component D1 is an example of a feature factor calculated by multivariate analysis. The calculation unit 14 extracts a feature factor (eg, first principal component D1) from a plurality of factors based on the first point cloud data P1 by multivariate analysis, and calculates the direction of the vector in this feature factor (eg, first principal component D1 direction) is calculated. The calculation unit 14 may extract a feature factor from a plurality of factors (eg, variables) based on the first point cloud data P1 by multivariate analysis other than principal component analysis, and calculate the direction of the vector in this feature factor. . In the present embodiment, processing for calculating the direction of the first principal component D1 by principal component analysis as multivariate analysis will be described as an example.

From the first point cloud data P1 stored in the storage unit 15, the first generation unit 12 generates a plane RS2A (eg, plane, see FIG. 5) that intersects the direction of the first principal component D1 calculated by the calculation unit 14. Extract each point in one or more second regions R2 (R2A, R2B, . . . , R2M) having second point cloud data P2A, P2B, . Execute the process to generate the . The plane RS2A is, for example, orthogonal to the direction of the first principal component D1. The second region R2 having a plane (eg, plane) is a region (eg, space) having a thickness in the direction of the first principal component D1. The second region R2 is, for example, a substantially flat surface, a cuboid, a cylinder, an elliptical cylinder, a polyhedron, or the like.

Note that the use of the plane RS2A is not limited to the plane RS2A, and any plane that intersects the direction of the first principal component D1 can be set. Or it may be a curved surface. The plane RS2A is set to have a width that covers the point group near the plane RS2A when viewed from the direction of the first principal component D1.

The second setting unit 113 executes second region setting processing for setting a second region R2 for the human body M for the first generating unit 12 to generate the second point cloud data P2. The second region R2 is a three-dimensional space. One second region R2 is smaller than the first region R1. However, the second region R2 may be the same as the first region R1. The second setting unit 113 sets the length (thickness in this case) of the second region R2 in the direction of the first principal component D1. The second region R2 is a three-dimensional space defined by the length in the direction of the first principal component D1 set by the second setting unit 113 from the plane RS2A.

The second setting unit 113 sets a new second region R2B at a position different in the direction of the first principal component D1 from the previously set second region R2A, and sets a new second region R2B at one end (one end side) of the first principal component D1. New second regions R2B, . That is, the second setting unit 113 sets a plurality of second regions R2A, R2B, . . . , R2M along the direction of the first principal component D1. The plurality of second regions R2A, R2B, . . . , R2M are regions having planes RS2A, RS2B, . The planes RS2A, RS2B, . In this way, the second setting unit 113 sets the new second region R2 (eg, , second region R2B).

The plurality of second regions R2A, R2B, . . . , R2M may have the same length in the direction of the first principal component D1, or may differ from each other. The second setting unit 113 may set the length of the plurality of second regions R2A and the like in the direction of the first principal component D1 to a predetermined length (eg, 10 mm), or the length of the first region R1 It may be set at a predetermined ratio (for example, 5% of the length of the first principal component D1) based on the length of the first principal component D1. The predetermined ratio may be stored in the storage unit 15 or set by the user through the input unit 4 . The second setting unit 113 sequentially arranges and sets the plurality of second regions R2A, R2B, . . . , R2M along the direction of the first principal component D1. The second setting unit 113 may set the plurality of second regions R2A, R2B, . good. Further, the second setting unit 113 may set a plurality of second regions R2A, R2B, . . . , R2M so as to overlap each other.

Also, one or all of the plurality of second regions R2 are analysis target regions. When all of the plurality of second regions R2 are analysis target regions, the second regions R2A, R2B, . ) subregion. Also, these first to Mth partial areas are the first to Mth divided areas or the first to Mth small areas. Second region data RD2 related to second region R2 set by second setting unit 113 may be stored in storage unit 15 .

, R2M set by the second setting unit 113, and generates second point cloud data P2 (P2A, P2B, . . . ). , P2M). The second point cloud data P2 may be stored in the storage unit 15 . Note that the second setting unit 113 may be included in the first generation unit 12 . Also, the second setting unit 113 may not be provided in the processing apparatus 1A. In this case, the first generator 12 may automatically set one or a plurality of predetermined ranges from the first point cloud data P1 as the second region R2.

The feature quantity calculation unit 114 executes feature quantity calculation processing for calculating a feature quantity from each of the plurality of second point cloud data P2 (P2A, P2B, . . . , P2M). The feature amount calculator 114, for example, calculates the shortest path as a feature amount from each second point cloud data P2.

FIG. 6 is a diagram showing an example of calculating the shortest path for the second point cloud data P2A. For example, the feature amount calculation unit 114 maps a point group (eg, a plurality of point data) existing in the second point group data P2A in the direction of the first principal component D1, and maps the point group in the direction of the first principal component D1. Point cloud data on an orthogonal virtual plane, and the shortest path is calculated by a method of fitting a predetermined shape (fitting of a predetermined shape) to the point cloud developed on this virtual plane, or a method such as the least squares method. (See FIG. 7). The predetermined shape is, for example, a shape that changes continuously. The predetermined shape may be, for example, an ellipse, an oval, a circle, or a curved shape, or a combination of these shapes.

The feature amount calculation unit 114 calculates the length of the shortest path (shortest path length) calculated by the method described above. The shortest path length calculated by the feature quantity calculation unit 114 may be stored in the storage unit 15 . Note that the feature amount calculation unit 114 is not limited to calculating the shortest path as the feature amount. The feature amount calculation unit 114 may calculate, as the feature amount, for example, an area surrounded by a point group developed on a virtual plane, or a volume of space surrounded by the second point group data P2A. good. Further, the feature amount calculation unit 114 can apply the following method as a modified example instead of the ellipse estimation (method of fitting an elliptical shape). For example, the feature amount calculation unit 114 obtains a convex hull from the second point cloud data P2 (eg, P2A), interpolates the points included in this convex hull with a B-spline curve, and obtains the second point cloud data P2. A feature amount (eg, perimeter length) may be calculated. Here, the convex hull is the minimum convex set containing each point given by the second point cloud data P2. The convex hull obtained from the second point cloud data P2 is the intersection of all convex sets including the points included in the second point cloud data P2, or the convex combination of the points included in the second point cloud data P2. It can be defined as a set. A convex set means, for example, that for any two points included in an object (second point cloud data P2 in this embodiment), any point on the line segment connecting those two points is also included in the object. Say. A convex combination is, for example, a linear combination of points (eg, vectors, scalars) that have non-negative coefficients that sum to one. A B-spline curve is, for example, a smooth curve defined from a plurality of given points and knot vectors.

Here, the feature amount calculator 114 may calculate the position of the first principal component D1 on the virtual plane. For example, if the point group developed on the virtual plane is subjected to fitting processing using an elliptical shape, the feature amount calculation unit 114 calculates the center of the elliptical shape as the position of the first principal component D1. good. In addition, it can be estimated that the shortest path length calculated by the feature quantity calculation unit 114 corresponds to the length of the circumference of that part of the human body M. FIG.

Also, the data amount of the second point cloud data P2 may be small. For example, when the position detection unit 2 acquires the reference point cloud data PF of the human body M, if the human body M is imaged from one direction, an image of the back side, which is a shadow in that direction, cannot be acquired. The reference point cloud data PF may be generated by the group data generator 22 without the point cloud on the back side.

FIG. 7 is a diagram showing another example of calculating the shortest path for the second point cloud data P2. Although the processing device 1A (or the posture analysis system SYS) shown in FIG. 1 has two (plural) detection units 21, the number of the detection units 21 may be one. In such a processing device 1A (or posture analysis system SYS), when the human body M is imaged from one detection unit 21 with the -Z direction (eg, see FIG. 1) as a viewpoint, the surface side of the human body M (eg, +Z side) ) is imaged, but the back side of the human body M (for example, the −Z side) is not imaged. Therefore, the reference point cloud data PF (in this case, there is no point cloud on the back side of the human body M) is generated by the reference point cloud data generation unit 22, and based on this reference point cloud data PF, the second point cloud data P2A1 ( 7) is extracted, the point cloud on the back side remains absent. However, it is possible to assume that the major parts of the human body M (eg, head, neck, torso, legs, arms) have a substantially elliptical profile on a plane orthogonal to the longitudinal direction. Therefore, the feature amount calculation unit 114 may calculate the shortest path by performing fitting processing on the second point cloud data P2A1 with an elliptical shape (see FIG. 7). Further, the feature amount calculation unit 114 may calculate the center of the elliptical shape as the position of the first principal component D1A. Further, when there is no point cloud on the back side as shown in FIG. 7, the detection unit 21 The human body M is imaged by the reference point cloud data generation unit 22 to generate point cloud data in the second direction, and the reference point cloud data generation unit 22 generates point cloud data (for example, 7) and the point cloud data with the second direction as the viewpoint may be combined (for example, merged) to generate the reference point cloud data PF. The reference point cloud data PF synthesized in this way is, for example, like the second point cloud data P2A shown in FIG. and have a point cloud at

Then, the determination unit 115 determines two features calculated by the feature amount calculation unit 114 based on the second point cloud data P2 (eg, P2A and P2B) of the adjacent second regions R2 (eg, R2A and R2B). Quantities (eg, shortest path length) are compared to determine whether or not the amount of change exceeds a predetermined value.

FIG. 8 is a graph showing the relationship between the position in the direction of the first principal component D1 and the shortest path length. For example, the determination unit 115 creates a graph in which the horizontal axis is the first principal component D1 and the vertical axis is the shortest path length (see FIG. 8), and determines whether the amount of change in the feature amount exceeds a predetermined value. . This predetermined value may be stored in the storage unit 15 in advance. The determination unit 115 may determine whether or not the first derivative changes, instead of determining whether or not the amount of change exceeds the predetermined value.

The determination unit 115 extracts the points PA and PB determined that the amount of change exceeds a predetermined value or that the primary differential changes as feature points (eg, points of change) (see FIG. 8). In the direction of the first principal component D1, the determination unit 115 extracts, as points PA and PB, portions in which the shortest path length changes significantly, or portions in which the shortest path length changes from increasing to decreasing or from decreasing to increasing. Points PA and PB extracted by determination unit 115 are stored in storage unit 15 .

When the determination unit 115 determines that the amount of change exceeds a predetermined value or that the first derivative changes, the resetting unit 116 first resets one or both of the adjacent second regions R2 used for the determination. A resetting process of setting a new first region R11 is executed for a range in which the amount of change does not exceed a predetermined value after being excluded from the set first region R1. The resetting unit 116 can set a new first region R11 having a narrower range than the previous first region R1 by this resetting process.

FIG. 9 is a diagram in which the second region R21 is set based on the newly set first region R11. The resetting unit 116, for example, excludes the second regions R2 outside the points PA and PB among the plurality of second regions R2, The range in the direction of the first principal component D1 is set as a new first region R11 (see FIGS. 8 and 9).

Then, when the resetting unit 116 sets the new first region R11, the generation unit 11 extracts the points in the first region R11 from the reference point cloud data PF to generate the first point cloud data P11. The calculator 14 calculates a first principal component D11 from the first point cloud data P11 by principal component analysis (see FIG. 9). The second setting unit 113 sets one or more second regions R21 (R21A, . . . , R21E) along the direction of the first principal component D11 (see FIG. 9).

The first generator 12 extracts each point existing in the second regions R21A, . . . , R21E to generate second point cloud data P21A, . The feature amount calculator 114 calculates the shortest path length, which is a feature amount, from the second point cloud data P21A and the like. The determination unit 115 determines whether or not the amount of change in the feature amount in the adjacent second regions R21 exceeds a predetermined value. When the determining unit 115 determines that the predetermined value is exceeded, the resetting unit 116 newly sets the first area as described above, and the above processing is repeated.

FIG. 10 is a graph showing the relationship between the position in the direction of the first principal component D11 and the shortest path length for the new second region R21. Note that the first principal component D11 corresponds to the range sandwiched between PA and PB with respect to the first principal component D1 shown in FIG. When the determination unit 115 determines that the amount of change in the feature amount in the adjacent second regions R21 does not exceed a predetermined value (see FIG. 10), the second point cloud data P21A, . . . , P21E and the first principal component The direction and position of D11 may be stored in the storage unit 15. FIG. Also, in the new first region R11, it is estimated that the change in the shortest path length along the first principal component D11 is small, or that the shortest path length does not change from an increase to a decrease or from a decrease to an increase. (See FIG. 10).

Note that in the above-described repeated processing from the generation unit 11 to the resetting unit 116, the determination unit 115 may determine the amount of change in the feature amount using the same predetermined value, or using a different predetermined value. You can judge. For example, the determination unit 115 may determine using a predetermined value that is smaller than the predetermined value used immediately before. One or both of the feature amount calculation unit 114 and the determination unit 115 may be included in the second setting unit 113 or the first generation unit 12 . Also, one or both of the feature amount calculation unit 114 and the determination unit 115 may not be provided in the processing device 1A. In this case, the first generator 12 may only generate the second point cloud data P2 from one or more second regions R2. Also, the resetting unit 116 may be included in the first setting unit 111 or the generating unit 11 . If the determination unit 115 is not provided in the processing apparatus 1A, the resetting unit 116 may not be provided in the processing apparatus 1A.

The second generation unit 13 performs a generation process of generating one or both of the three-dimensional model of the human body M and posture information of the human body M based on the second point cloud data P2 generated by the first generation unit 12 . The second generation unit 13 includes a posture estimation unit 117, a model generation unit 118, and a rendering processing unit 119, as shown in FIG. The posture estimation unit 117 performs posture estimation processing for estimating the posture of the human body M based on the second point cloud data P2 generated by the first generation unit 12 . The posture estimation unit 117 generates, for example, position information of a target portion (eg, analysis target portion, target region) in the second region R2 (R2A, etc.) of the human body M. FIG.

The posture estimation unit 117 performs, for example, recognition processing (eg, pattern recognition, shape recognition, skeleton recognition) on the shape of the target portion (target part) of the human body M obtained from the second point cloud data P2. Posture estimation section 117 generates the position information of the target portion by recognition processing. The positional information of the target portion includes, for example, coordinates (eg, three-dimensional coordinates) of a point representing the target portion. Posture estimation section 117 calculates the coordinates of the point representing the target portion by the recognition process described above. Posture estimation section 117 causes storage section 15 to store the coordinates of the point representing the target portion. The posture estimation unit 117 uses the second point cloud data P2 after the principal component analysis, and therefore can estimate the target portion with high accuracy.

Posture estimation section 117 obtains, as a result of the recognition process, the position of the target portion, information on the target portion (eg, attribute information, name (eg, thigh, upper arm)), and two target portions (eg, thigh and lower leg). , the portion from the ankle to the tip of the foot, and the upper arm and the forearm), posture information (eg, skeleton information, skeleton data) is generated as a set. Posture estimation section 117 causes storage section 15 to store the posture information.

The posture estimation unit 117 estimates the posture of the human body M based on the posture information. The posture estimation unit 117 calculates the relative position (eg, angle) between a first line connecting a pair of target portions having a connection relationship and a second line connected to the first line and connecting a pair of characteristic portions having a connection relationship. , the pose of the portion containing the first and second lines is estimated. The posture estimation unit 117 estimates the posture of a part or the whole of the human body M with high accuracy by estimating the posture of the target part as described above.

Posture estimation section 117 may refer to posture definition information that defines the posture of human body M or the like to estimate the posture. The posture definition information is, for example, a set of information representing the type of posture and information defining the relative position of each target portion. The information indicating the type of posture is, for example, the name of the posture such as standing posture or sitting posture. The posture estimation unit 117 may estimate (identify) the posture of the human body M by comparing the posture information and the posture definition information.

In addition, the posture estimation unit 117 calculates the connection portion (human body M joint position) may be estimated. The multiple regions of interest in the object are parts connected to each other across joints (eg, head and neck, shoulders and upper arms, upper arms and lower arms, buttocks and thighs, thighs and lower legs).

FIGS. 11A and 11B are diagrams showing an example of estimating the posture of an object (human body M in this embodiment) by the second generator 13 (eg, posture estimator 117) according to this embodiment. . An example in which the posture estimating unit 117 estimates joint positions in the human body M will be described with reference to FIG. The posture estimation unit 117 calculates the first principal component when setting the second region R21 with respect to the first part Q1 (in this case, the thigh), which is a part (eg, target region) of the object (the human body M in this embodiment). Based on D11 and the first principal component D111 when setting the second region R121 with respect to the second part Q2 (lower leg in this case) which is another part of the human body M and different from the first part Q1, Information about the joint K, which is the connecting portion between the first part Q1 and the second part Q2, is generated. After the posture (eg, position, direction, length) of the "thigh" of the human body M is estimated with high accuracy as the target part (first part Q1) as in the above embodiment, other The first region is set so as to include the "lower leg", which is the target part (second part Q2) of the above, as an outline feature. This first region is a part of the first region R1 (or a new first region R11) for the "thigh", or a part of the first region R1 excluding the first region R11. or all of them. Also, the first region including the "lower leg" may be, for example, the first region R1 shown in FIG. Then, when the first region is set, the processing of calculating the first principal component, setting the second region, generating the second point cloud data, and calculating the feature amount is repeated in the same manner as in the above-described embodiment. After that, as shown in FIG. 11A, the first principal component D111 is calculated from the first region R111 with the “lower leg” as the target region (second region Q2), and the first principal component D111 generated from the second region R121 is calculated. By using the two-point cloud data, the posture (eg, position, direction, length) of the “lower leg”, which is the second part Q2, is estimated by the posture estimation unit 117 with high accuracy.

Next, as shown in FIG. 11B, the posture estimation unit 117 determines the first principal component D11 when the second region R21 is set with the "thigh" (first part Q1) as the target part, and " The first principal component D111 when the second region R121 is set with the "lower leg" (second region Q2) as the target region is virtually extended. The first principal component D11 and the first principal component D111 may be along parallel and spaced planes that do not intersect each other, or may be in a twisted positional relationship. Then, posture estimation section 117 calculates line segment LX so that the distance between first principal component D11 and first principal component D111 is the shortest, and determines the midpoint between "thigh" (first part Q1) and " It is estimated as a joint K (knee) that is a connection portion with the lower leg (second part Q2). In addition, when the first principal component D11 and the first principal component D111 are extended and the first principal component D111 and the first principal component D111 are extended, the posture estimation unit 117 estimates the intersection point as the joint K (knee) when the two intersect. Note that the posture estimation unit 117 is not limited to estimating the joint K as the midpoint of the line segment LX. The posture estimation unit 117 may estimate a joint K other than the midpoint of the line segment LX. Alternatively, a point obtained by dividing the line segment LX at a predetermined ratio (eg, 3:7) may be estimated as the joint K, or a point on the first principal component D11 or a point on the first principal component D111. may

Further, the model generation unit 118 executes model generation processing for generating model information of the human body M based on the posture information. The model information is, for example, three-dimensional CG model data, and includes shape information of the human body M. The model generator 118 calculates model information of the human body M based on the second point cloud data P2 generated by the first generator 12 . The model generation unit 118 executes surface processing for calculating surface information as shape information. Surface information includes, for example, at least one of polygon data, vector data, and draw data. The surface information includes coordinates of multiple points on the surface of the human body M and connection information between the multiple points. The connection information (eg, attribute information) includes, for example, information that associates points at both ends of lines corresponding to ridges (eg, edges) on the surface of the human body M with each other. The connection information also includes, for example, information that associates a plurality of lines corresponding to the contours of the surface of the human body M with each other.

In the surface processing, the model generator 118 estimates a plane between a point selected from a plurality of points included in the second point group data P2 and its neighboring points. In the surface processing, the model generation unit 118 converts the second point cloud data P2 into polygon data having plane information between points. The model generation unit 118 converts the point cloud data into polygon data by, for example, an algorithm using the least squares method. The model generation unit 118 stores the calculated surface information in the storage unit 15 .

Note that the model information may include texture information of the human body M. FIG. The model generator 118 may generate surface texture information defined by three-dimensional point coordinates and related information. The texture information includes, for example, at least one information of characters, figures, patterns, textures, patterns, irregularities on the surface of the object, specific images, and colors (eg, chromatic and achromatic colors). The model generation unit 118 may store the generated texture information in the storage unit 15 .

The model information may also include image spatial information (eg, lighting conditions, light source information). The light source information includes the position of the light source that irradiates the human body M with light (eg, illumination light), the direction in which the light is radiated from the light source to the human body M (irradiation direction), the wavelength of the light radiated from the light source, and at least one item of information among the types of this light source. The model generation unit 118 may calculate light source information using, for example, a model that assumes Lambertian reflection, a model that includes Albedo estimation, or the like. The model generation unit 118 may cause the storage unit 15 to store the generated spatial information. The model generator 118 may not generate one or both of texture information and spatial information.

The rendering processing unit 119 includes, for example, a graphics processing unit (GPU). Note that the rendering processing unit 119 may be configured such that the CPU and memory execute each process according to an image processing program. The rendering processing unit 119 performs, for example, at least one of drawing processing, texture mapping processing, and shading processing.

In the drawing process, the rendering processing unit 119 can calculate an estimated image (eg, reconstructed image) of the shape defined in the shape information of the generated model information viewed from an arbitrary viewpoint. The rendering processing unit 119 can reconstruct a model shape (eg, estimated image) from model information (eg, shape information) by, for example, drawing processing. The rendering processing unit 119 may store the calculated estimated image data in the storage unit 15 .

In the texture mapping process, the rendering processing unit 119 can calculate an estimated image in which, for example, an image indicated by texture information of model information is pasted on the surface of an object on the estimated image. The rendering processing unit 119 can calculate an estimated image in which a texture different from that of the target object is applied to the surface of the object on the estimated image. In the shading process, the rendering processing unit 119 can calculate an estimated image in which, for example, shadows formed by the light source indicated by the light source information of the model information are added to the object on the estimated image. In the shading process, the rendering processing unit 119 can calculate an estimated image by adding a shadow formed by an arbitrary light source to an object on the estimated image, for example.

Note that the second generation unit 13 does not have to include the posture estimation unit 117 , the model generation unit 118 , and the rendering processing unit 119 . At least one of the posture estimation unit 117, the model generation unit 118, and the rendering processing unit 119 may be provided inside the processing device 1A outside the second generation unit 13. FIG. Moreover, at least one of the posture estimation unit 117, the model generation unit 118, and the rendering processing unit 119 may be provided outside the processing device 1A and may be communicably connected to the processing device 1A.

The processing device 1A (or the posture analysis system SYS) displays the human body M displayed on the display unit 3 by the user operating the input unit 4 based on the information of the human body M generated by the second generation unit 13 described above. Viewpoints can be switched for a three-dimensional model or posture information. In addition, the processing device 1A (or the posture analysis system SYS) can measure the body shape based on the information of the human body M generated by the second generation unit 13, and score the body shape based on the measurement result. For example, the processing device 1A displays on the display unit 3 a score representing the measurement result of the body shape of the human body M measured based on the information of the human body M (for example, the posture information including the point group, etc.) as an absolute or relative numerical value. can be displayed. This score is displayed on the display unit 3 using a table format or a graph format in chronological order (for example, the horizontal axis indicates time or date). Note that the processing device 1A causes the display unit 3 to display the result of comparing the score based on the information on the user's human body M and the score based on the information on the teacher human body M stored in the storage unit 15 in advance. Alternatively, the display unit 3 may display the result of comparing the user's past score stored in the storage unit 15 and the currently measured score. The information on the user's human body M and the information on the teacher's human body M may be superimposed and displayed on the display unit 3 . A plurality of pieces of information on the human body M serving as a teacher are stored in the storage unit 15 so that information on the teacher based on the user's proficiency level can be used. The processing device 1A can change the information of the human body M serving as the teacher based on the proficiency level of the user. In addition, the display unit 3 can display the outer circumference dimension (eg, the length around the thigh) of each part (eg, thigh, upper arm) of the human body M. FIG. Note that the shortest path length calculated by the feature quantity calculation unit 114 may be used as the outer circumference dimension of each part. When displaying the dimensions of each part of the human body M, the dimensions may be displayed in chronological order, or a three-dimensional model of a part or the whole of the human body M may be displayed on the display unit 3 in chronological order.

Next, a processing method according to this embodiment will be described. FIG. 12 is a flow chart showing an example of a processing method according to this embodiment. 1 to 10 described above will be used as appropriate when explaining the flowchart of FIG. The processing device 1A (or the posture analysis system SYS) acquires reference point cloud data PF of an object (in this case, a human body M) (step S1). In step S1, the reference point cloud data PF may be collectively sent from the position detection unit 2 as one file to the processing device 1A, or may be divided as a plurality of files and sequentially sent to the processing device 1A. .

Subsequently, the first setting unit 111 of the processing device 1A sets the first region R1 from the reference point cloud data PF, and the generation unit 11 extracts the points in the first region R1 to generate the first point cloud data P1. is generated (step S2). In step S2, an image relating to the set size and position of the first region R1 may be displayed on the display unit 3 in a state of being superimposed on the reference point cloud data PF. Further, in step S2, the first point cloud data P1 may be displayed on the display unit 3 in a state separated from the reference point cloud data PF, or the portion of the first point cloud data P1 of the reference point cloud data PF may be displayed. It may be displayed on the display unit 3 with the color changed from that of other parts, or with the part of the first point cloud data P1 blinking. As described above, the first region R1 is roughly cut out from the reference point cloud data PF. The first region R1 is set, for example, as a wide region that includes the site desired by the user to be analyzed.

Subsequently, the calculator 14 calculates the direction of the first principal component D1 of the first point cloud data P1 by principal component analysis (step S3). In step S3, the calculator 14 may calculate the position of the first principal component D1 viewed from the direction of the first principal component D1 and the length of the first principal component D1. Then, the second setting unit 113 sets a second region R2 (R2A) having a plane (eg, plane RS2A) perpendicular to the first principal component D1 (step S4). In step S4, an image relating to the set size and position of the second region R2 may be displayed on the display unit 3 while being superimposed on the first point cloud data P1 or the reference point cloud data PF.

Subsequently, the first generator 12 extracts points in the second region R2 from the first point cloud data P1 to generate the second point cloud data P2 (step S5). In step S5, the second point cloud data P2 may be displayed on the display unit 3 in a state separated from the first point cloud data P1 or the reference point cloud data PF. The second point cloud data P2 portion of the PF may be displayed on the display unit 3 in a different color from the other portions, or in a state in which the second point cloud data P2 portion is blinked.

Subsequently, the second generator 13 generates information on the human body M from the second point cloud data P2 (step S6). In step S5, the second generation unit 13 may cause the display unit 3 to display one or both of the three-dimensional model of the human body M and the posture information as the information of the human body M.

The three-dimensional model and posture information (eg, skeletal information, skeleton data) may be switched by the user's operation of the input unit 4 so that either one of them is displayed on the display unit 3. The posture information may be displayed side by side on the display unit 3 . In addition, for each part of the human body M, numerical values such as external dimensions may be displayed on the display unit 3 together with the three-dimensional model, or specific external dimensions may be displayed on the display unit 3 together with changes over time. may be displayed.

13 and 14 are flowcharts showing another example of the processing method according to this embodiment. In this flow chart, steps S1 to S5 are substantially the same as the processing shown in FIG. 12, so description thereof will be omitted or simplified. It should be noted that in step S4 of the flowchart of FIG. 13, the second setting unit 113 sets a plurality of second regions R2. For example, in step S4, the second setting unit 113 sequentially sets a plurality of second regions R2 along the direction of the first principal component D1.

The second setting unit 113 sets the length (thickness in this case) of the second region R2 in the direction of the first principal component D1. Since the number of second regions R2 increases, the subsequent processing load increases from the viewpoint of data amount. Therefore, the second setting unit 113 may set the number of the second regions R2 or the length in the direction of the first principal component D1 based on the processing capability of the processing device 1A.

Following step S5, the feature quantity calculator 114 calculates the shortest path length as a feature quantity from the second point cloud data P2 (step S11). Subsequently, the first generation unit 12 or the feature amount calculation unit 114 determines whether the second region R2 of the second point cloud data P2 for which the shortest path length has been calculated is the last second region R2 (step S12). . If it is not the last second region R2 (NO in step S12), the process returns to step S4, and the second setting unit 113 sets the second region R2 next to the previously set second region R2. If it is the last second region R2 (YES in step S12), the determining unit 115 compares the shortest path lengths in the adjacent second regions R2 and extracts the amount of change (step S13).

Subsequently, the determination unit 115 determines whether or not the amount of change in the shortest path length exceeds a predetermined value (step S14). If the amount of change exceeds the predetermined value (NO in step S14), the resetting unit 116 excludes one or both of the adjacent second regions R2 used for the determination from the previously set first region R1, A range in which the amount of change does not exceed a predetermined value is set as a new first region (step S15). Then, the processing from step S2 to step S14 is repeated. Through this series of processing, portions with large variations in the shortest path length are excluded, and the shape or posture of the user-desired part (analysis target region) can be estimated with high robustness and high accuracy.

If the amount of change does not exceed the predetermined value (YES in step S14), the second generator 13 generates information on the human body M from the second point cloud data P2 (step S16). Since this step S16 is the same as step S6 shown in FIG. 12, description thereof is omitted.

In addition, in the flowcharts shown in FIGS. 13 and 14, in the processing from step S5 to step S12, the shortest path length is calculated by sequentially setting the plurality of second regions R2, but the processing is not limited to this. For example, in step S5, the second setting unit 113 sets a plurality of second regions R2, and in step S6, the first generation unit 12 extracts point groups (point data) from each of the plurality of second regions R2. to generate a plurality of second point cloud data P2, and in step S11, the feature amount calculation unit 114 performs calculation of the shortest path length, such as calculating the shortest path length from each of the plurality of second point cloud data P2. You may carry out in parallel with respect to several 2nd point-group data P2.

[Second embodiment]
Next, a second embodiment will be described. FIG. 15 is a diagram illustrating an example of a processing device according to the second embodiment; In this embodiment, the same reference numerals are given to the same configurations as those of the above-described embodiment, and the description thereof will be omitted or simplified. In the processing device 1B shown in FIG. 15, a position detection unit 2 (for example, a point group acquisition unit) is provided (built in) inside the processing device 1B. This processing device 1B includes a function of acquiring reference point cloud data PF in one unit (package). For example, a notebook computer is used as the processing device 1B, and a camera mounted on the notebook computer is used as the detection unit 21 .

The processing device 1B does not require a separate position detection unit 2, and is convenient when being carried. In addition, in the processing device 1B, the storage unit 23 of the position detection unit 2 may also be used as the storage unit 15 included in the processing device 1B. That is, the storage unit 23 may not be provided in the processing device 1B.

[Third embodiment]
Next, a third embodiment will be described. FIG. 16 is a diagram illustrating an example of a processing device according to the third embodiment; In this embodiment, the same reference numerals are given to the same configurations as those of the above-described embodiment, and the description thereof will be omitted or simplified. In the processing device 1C shown in FIG. 16, the reference point cloud data generation unit 22 and the storage unit 23 of the position detection unit 2 (for example, the point cloud acquisition unit) are provided (built in) inside the processing device 1C. And the detection part 21 is a mode provided in the exterior of 1 C of processing apparatuses. The detection unit 21 may have a configuration capable of capturing an image of the human body M (object), and may be, for example, a camera or a mobile terminal (eg, smart phone, tablet) having an image capturing function. The detection unit 21 outputs the depth map as the position information of the human body M to the processing device 1C (reference point cloud data generation unit 22).

The processing device 1C can easily generate the reference point cloud data PF by acquiring the image of the human body M from the detection unit 21 only by preparing the detection unit 21 capable of imaging the object (in this case, the human body M). A notebook computer or desktop computer without an imaging function can be used. In addition, in the processing device 1C, the storage unit 23 of the position detection unit 2 may also be used as the storage unit 15 included in the processing device 1C. That is, the storage unit 23 may not be provided in the processing device 1C.

Next, screens and the like displayed on the display unit 3 in this embodiment will be described. 17 to 23 are diagrams showing examples of screens displayed on the display unit 3 in the first to third embodiments. 17 to 23 exemplify screens of a body type measurement application when the above-described embodiment is applied to body type measurement of the human body M, for example. FIG. 17 shows the start screen SA of the body measurement application. Icons such as "Measurement", "User Information", "History", "Configuration", and "Close" are displayed on the start screen SA, allowing the user to perform desired operations. Note that the user name is "test" in this body measurement application.

When the user selects the "Measurement" icon on the start screen SA, the measurement screen SB is displayed as shown in FIG. 18. If "DeviceStatus" is OK, the position detection unit 2 shows a state in which the human body M can be imaged by the detection unit 21 of No. 2. FIG. When "DeviceStatus" is OK, an image picked up by the position detection unit 2 is displayed below the icons such as "Device1" on the measurement screen SB. Further, when the "DeviceStatus" is NG, the measurement screen SB of FIG. 18 is displayed. When a plurality of position detectors 2 or detectors 21 are used, the positions of the position detectors 2, model names, etc. may be displayed below the icons "Device1", "Device2", and so on. It should be noted that the above processing device causes the display unit 3 to display images picked up by the respective position detection units 2 below the icons "Device1", "Device2", etc., regardless of the state of "DeviceStatus". good too. Further, the above processing device may display a teaching image for teaching the direction in which the human body M should be imaged by the position detection unit 2 or the like on the measurement screen SB.

An image of the posture required of the human body M at the time of imaging is displayed below the "PoseModel" icon on the measurement screen SB. The user can easily grasp the posture at the time of imaging by referring to the image of this posture. In addition, the user can change the posture displayed on the measurement screen SB by operating the “PoseModel” icon or the posture image. At the time of change, icons such as "PoseModel1", "PoseModel2", etc., and pose images associated with the icons are displayed.

In FIG. 18, when the user selects the "Capture1" icon on the measurement screen SB, for example, the position detection unit 2 or the detection unit 21 shown in "Device1" captures an image of the human body M, and from the captured data, the reference point cloud is detected. The data generator 22 generates reference point cloud data PF. The generated reference point cloud data PF may be displayed below the "Device1" icon on the measurement screen SB.

When the user selects the "User Information" icon on the start screen SA, a user management screen SC is displayed as shown in FIG. 19, and user information can be registered, edited, and deleted. For example, when registering user information, a new user registration screen SC1 window as shown in FIG. 20 can be displayed. A new user can manage various data of this body measurement application by inputting predetermined items in the window of the new user registration screen SC1.

When the user selects the "History" icon on the start screen SA, a data history screen SD including a list of past measurement results is displayed as shown in FIG. On the data history screen SD, the perimeter dimensions of the parts of the human body M for the user are displayed in association with the date of measurement. In the data history screen SD shown in FIG. 21, the outer circumference dimensions of "LeftUpperArm" and "Waist" are displayed in association with the date of measurement. The outer circumference dimensions of "LeftUpperArm" and "Waist" are the second point cloud data when the second region R2 is "LeftUpperArm" (upper left arm) and "Waist" (waist, trunk) in the above description of the embodiment. It is obtained from the 3D model generated from P2. That is, in the present embodiment, since a three-dimensional model is generated from the second point cloud data P2 subjected to principal component analysis, the outer peripheral dimensions of "LeftUpperArm" and "Waist" are accurately measured.

When the user selects the "Waist" icon on the data history screen SD, a measurement position adjustment screen SD1 is displayed as shown in FIG. The measurement position adjustment screen SD1 is a screen for adjusting (changing) the numerical values displayed on the data history screen SD. On the measurement position adjustment screen SD1, a plurality of positions in the + direction (upward) and - direction (downward) from the reference position (+0) and the outer circumference dimensions at the positions are displayed side by side in "Position Result". The reference position (+0) is selected on the measurement position adjustment screen SD1 shown in FIG. The user can adjust (change) the numerical values displayed on the data history screen SD by selecting icons at other positions (eg, +2, -3). In addition, on the measurement position adjustment screen SD1, a horizontal direction line H indicating the selected position on the image is displayed. The horizontal pointing line H moves according to the position selected by the user. In the measurement position adjustment screen SD1, a vertical indicator line V indicating the vertical direction at the center of the human body M is displayed.

Subsequently, when the user selects the "Configuration" icon on the start screen SA, a camera position adjustment screen SE is displayed as shown in FIG. On the camera position adjustment screen SE, for example, various settings of the device (for example, the position detection unit 2) such as setting of the data storage location and adjustment of the camera position can be performed.

As described above, according to the present embodiment, a three-dimensional model or posture information can be generated with high accuracy for part or all of an object (eg, human body M). When the object is the human body M, the three-dimensional shape of each part of the human body M is generated with high accuracy, so the size (perimeter dimension) of each part of the human body M can be obtained easily and accurately. Further, according to the present embodiment, since the posture information of each part of the human body M is generated with high accuracy, the posture of each part of the human body M (eg, thighs) during exercise (eg, ballet, baseball, golf) can be generated. orientation, upper arm orientation, ballet form, pitching form, batting form, golf swing form) can be obtained easily and accurately.

Note that the processing device 1A (or the posture analysis system SYS) described above may include two or more position detection units 2 . Also, the posture analysis system SYS may include two or more processing devices 1A, 1B, and 1C.

In the above-described embodiments, the generation unit 11, the first generation unit 12, the second generation unit 13, the calculation unit 14, the first setting unit 111, the feature extraction unit 112, and the second setting unit 113 of the processing apparatuses 1A, 1B, and 1C , the feature amount calculation unit 114, the determination unit 115, the resetting unit 116, the posture estimation unit 117, the model generation unit 118, the rendering processing unit 119, and the reference point cloud data generation unit 22 of the position detection unit 2 are executed by the computer. By being read, it may be realized as concrete means in which software and hardware resources cooperate.

Also, in the above-described embodiments, the processing apparatuses 1A, 1B, and 1C include, for example, computer systems. The processing apparatuses 1A, 1B, and 1C read the processing program stored in the storage unit 15 and execute various processing according to the processing program. This processing program, for example, causes the computer to obtain the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object, and obtains the direction of the first principal component from the first point cloud data. generating second point cloud data by extracting points in an area intersecting with the direction of , and generating object information from the second point cloud data. This program may be provided by being recorded on a computer-readable storage medium (eg, non-transitory tangible media).

Although the embodiments have been described, the technical scope of the present invention is not limited to the contents described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. Also, the requirements described in the above-described embodiments and the like can be combined as appropriate. Also, the execution order of each process shown in the embodiment can be implemented in any order as long as the output of the previous process is not used in the subsequent process. Further, even if the processing in the above-described embodiment is described using “first”, “next”, “following”, “and”, etc. for convenience, it is not essential to perform the processing in this order. In addition, as long as it is permitted by laws and regulations, the disclosure of all the documents cited in the above-described embodiments and the like is used as part of the description of the text.

It should be noted that the technical scope of the present invention is not limited to the aspects described in the above embodiments and the like. One or more of the requirements described in the above embodiments and the like may be omitted. Also, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, as long as it is permitted by law, the disclosure of Japanese Patent Application No. 2018-143688 and all the documents cited in the above-described embodiments and the like are incorporated into the description of the text.

D1, D11 First principal component M Human body (object) K Joint (connection part) LX Line segment P1, P11 First point cloud data P2, P2A to P2M・Second point cloud data PF Reference point cloud data Q Outline feature Q1 First part Q2 Second part R1, R11 First area R2, R2A, R2B~ R2M, R21A to R21E Second area RS2A Plane (surface) SYS Attitude analysis system 1A, 1B, 1C Processing device 2 Position detection unit (point cloud acquisition unit) 11 Generating unit 12 First generating unit 13 Second generating unit 14 Calculating unit 22 Reference point cloud data generating unit 111 First setting unit 112 Features Extraction unit 113 Second setting unit 114 Feature amount calculation unit 115 Determination unit 116 Reset unit

Claims (25)

a calculation unit that obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
a first generation unit that generates second point cloud data by extracting points in a region that intersects the direction of the first principal component from the first point cloud data;
a feature amount calculation unit that calculates feature amounts at a plurality of positions in the direction of the first principal component from the second point cloud data;
a resetting unit that resets the range of the first point cloud data based on a change in the direction of the first principal component of the feature amount;
a second generation unit that generates information about the object from the second point cloud data ;
The processing device , wherein the calculation unit obtains the direction of the first principal component from the first point cloud data included in the range of the first point cloud data reset by the resetting unit. The first point cloud data is data of a range smaller than a range including all point data based on position information of each point on the object,
2. The processing device of claim 1, wherein one of said regions containing said second point cloud data is smaller than the extent of said first point cloud data. 3. The processing apparatus according to claim 1, wherein said region is a partial region having a plane perpendicular to the direction of said first principal component. The process according to any one of claims 1 to 3, wherein the second generating unit generates information regarding at least one of a posture and a shape of a portion of the object corresponding to the second point cloud data. Device. The processing apparatus according to any one of claims 1 to 4, comprising a first setting unit that sets a range of the first point cloud data. Having a feature extraction unit for extracting the external shape feature of the object,
6. The processing device according to claim 5, wherein said first setting unit sets the range of said first point cloud data including said outline feature. The processing apparatus according to any one of claims 1 to 6, further comprising a second setting section that sets the area. 8. The processing device according to claim 7, wherein said second setting unit sets the length of said region in the direction of said first principal component. The second setting unit sets the plurality of regions at mutually different positions in the direction of the first principal component,
9. The processing device according to claim 7, wherein said first generation unit extracts a plurality of points within said region to generate said second point cloud data. The second generation unit according to any one of claims 1 to 9, wherein the second generation unit generates at least one of a three-dimensional model of the object and posture information of the object based on the second point cloud data. processing equipment. The processing device according to any one of claims 1 to 10, wherein the feature quantity calculation unit calculates a shortest path of the second point cloud data as the feature quantity. a calculation unit that obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
a first generation unit that generates second point cloud data by extracting points in a region that intersects the direction of the first principal component from the first point cloud data;
a feature amount calculation unit that calculates a feature amount from the second point cloud data;
a second generation unit that generates information about the object from the second point cloud data;
The processing device, wherein the feature amount calculation unit calculates a shortest path of the second point cloud data as the feature amount. 13. The processing apparatus according to claim 12 , wherein said feature amount calculation unit calculates said shortest path by applying a predetermined shape to said second point cloud data. 14. The processing apparatus according to claim 13 , wherein said predetermined shape is one of an ellipse, an oval, a circle, and a curved shape. a position detection unit that detects the position information of each point on the object;
15. The processing apparatus according to any one of claims 1 to 14 , further comprising a point cloud data generation unit that generates point cloud data used to generate the first point cloud data based on the position information. a calculation unit that obtains the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
a first generation unit that generates second point cloud data by extracting points in a region that intersects the direction of the first principal component from the first point cloud data;
a second generation unit that generates information about the object from the second point cloud data;
The second generator generates the first principal component of the region set with respect to a first portion of the object and the first principal component of the region set with respect to a second portion different from the first portion of the object. a processing device that generates information about a connecting portion between the first portion and the second portion based on the components. The second generation unit generates a line segment that has the shortest distance between the first principal component of the region set with respect to the first portion and the first principal component of the region set with respect to the second portion. 17. The processing apparatus according to claim 16 , wherein the midpoint of is the connecting portion. a calculation unit that extracts a feature factor from a plurality of factors based on first point cloud data based on position information of each point on an object and obtains the direction of a vector in the feature factor;
a first generation unit that generates second point cloud data by extracting points in an area that intersects with the direction;
a second generation unit that generates information about the object from the second point cloud data;
The calculation unit obtains the direction of the vector from which the feature factor is extracted by principal component analysis,
The second generator is configured based on the feature factor of the region set for a first portion of the object and the feature factor of the region set for a second portion different from the first portion of the object. to generate information about a connecting portion between the first portion and the second portion. The second generation unit determines a midpoint of a line segment having the shortest distance between the feature factor of the region set for the first portion and the feature factor of the region set for the second portion. 19. The processing device according to claim 18 , being said connecting portion. A processing apparatus according to any one of claims 1 to 19 ;
A posture analysis system comprising: a point cloud acquisition unit that acquires point cloud data used to generate first point cloud data based on position information of each point on an object. Obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
generating second point cloud data by extracting points in a region intersecting the direction of the first principal component from the first point cloud data;
calculating feature amounts at a plurality of positions in the direction of the first principal component from the second point cloud data;
resetting the range of the first point cloud data based on a change in the direction of the first principal component of the feature quantity;
generating information of the object from the second point cloud data ;
In the calculation of the direction of the first principal component, the direction of the first principal component is obtained from the first point cloud data included in the reset range of the first point cloud data . to the computer,
Obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
generating second point cloud data by extracting points in a region intersecting the direction of the first principal component from the first point cloud data;
calculating feature amounts at a plurality of positions in the direction of the first principal component from the second point cloud data;
resetting the range of the first point cloud data based on a change in the direction of the first principal component of the feature quantity;
generating information of the object from the second point cloud data ;
A processing program , wherein in calculating the direction of the first principal component, the direction of the first principal component is obtained from the first point cloud data included in the reset range of the first point cloud data . to the computer,
Obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
generating second point cloud data by extracting points in a region intersecting the direction of the first principal component from the first point cloud data;
calculating a feature amount from the second point cloud data;
generating information of the object from the second point cloud data;
A processing program, wherein in calculating the feature amount, a shortest path of the second point cloud data is calculated as the feature amount. to the computer,
Obtaining the direction of the first principal component by principal component analysis of the first point cloud data based on the position information of each point on the object;
generating second point cloud data by extracting points in a region intersecting the direction of the first principal component from the first point cloud data;
generating information of the object from the second point cloud data;
In the generation of the information of the object, the first principal component of the area set with respect to the first part of the object and the first principal component of the area set with respect to a second part different from the first part of the object A processing program for generating information about a connecting portion between the first portion and the second portion based on principal components. to the computer,
Extracting a feature factor from a plurality of factors based on first point cloud data with position information of each point on the object, and determining the direction of the vector in the feature factor;
generating second point cloud data by extracting points in an area intersecting the direction;
generating information of the object from the second point cloud data;
In calculating the direction of the vector, the direction of the vector from which the feature factor is extracted is obtained by principal component analysis;
In the generation of information of the object, the feature factor of the region set for a first portion of the object and the feature factor of the region set for a second portion different from the first portion of the object. a processing program for generating information about a connecting portion between the first portion and the second portion based on the

Images