JPWO2020026677A1 - Detection device, processing device, detection method, and processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To discriminate between a portion on one side of an object and a portion on the other side with respect to an intersection direction intersecting the moving direction of the moving object. SOLUTION: A detection device has a detection unit that detects position information of each point of a moving object, and a position where a change amount of the position information satisfies a predetermined condition in an intersection direction intersecting the moving direction and the vertical direction of the object. A reference calculation unit that calculates with A discriminating unit for discriminating a second portion of an object arranged on the second side opposite to the first side with respect to the reference plane is provided.

Description

The present invention relates to a detection device, a processing device, a detection method, and a processing program.

As a technique for detecting an object, for example, there is a technique described in Patent Document 1 below. When detecting a moving object such as a walking human body, it may be difficult to distinguish between the left side portion and the right side portion with respect to the moving direction. For example, when detecting a human body during walking or running, the left foot and the right foot are at substantially the same position in the shoulder width direction of the human body, and it is difficult to distinguish between the left foot and the right foot.

Japanese Unexamined Patent Publication No. 2010-134546

According to the aspect of the present invention, a detection unit that detects the position information of each point of the moving object and a position where the amount of change in the position information satisfies a predetermined condition in the intersection direction intersecting the moving direction and the vertical direction of the object. A reference calculation unit that calculates with Provided is a detection device including a discriminating unit for discriminating a second portion of an object arranged on a second side opposite to the first side with respect to a reference plane.

According to the aspect of the present invention, the position where the amount of change in the position information of each point on the surface of the object satisfies a predetermined condition in the intersection direction intersecting the moving direction and the vertical direction of the moving object is calculated as a reference position. A reference calculation unit to be used, a first portion of the object arranged on the first side with respect to the reference plane including the moving direction and the vertical direction based on the reference position, and a first portion of the object with respect to the reference plane. A processing device including a discriminating unit for discriminating a second portion arranged on a second side opposite to the side is provided.

According to the aspect of the present invention, the position information of each point of the moving object is detected, and the position where the amount of change in the position information satisfies a predetermined condition in the intersection direction where the moving direction and the vertical direction of the object intersect. The first part of the object placed on the first side with respect to the reference plane including the movement direction and the vertical direction based on the calculation as the reference position and the first side with respect to the reference plane. A detection method including the determination of the second portion of the object arranged on the second side opposite to the above is provided.

According to the aspect of the present invention, the computer is based on a position where the amount of change in the position information of each point on the surface of the object satisfies a predetermined condition in the intersecting direction where the moving direction and the vertical direction of the moving object intersect. Calculated as a position, and based on the reference position, the first part of the object placed on the first side with respect to the reference plane including the moving direction and the vertical direction, and the opposite to the first side with respect to the reference plane. A processing program for discriminating from the second part of the object arranged on the second side of the object and executing the process is provided.

It is a figure which shows the detection apparatus which concerns on 1st Embodiment. It is a figure which shows the detection part which concerns on 1st Embodiment. It is a figure which shows the process of the point cloud data generation part which concerns on 1st Embodiment. (A) to (C) are diagrams showing the processing of the reference calculation unit according to the first embodiment. (A) and (B) are diagrams showing the processing of the reference calculation unit according to the first embodiment. It is a figure which shows the process of the partial discrimination part which concerns on 1st Embodiment. It is a figure which shows the process of the posture estimation part which concerns on 1st Embodiment. It is a flowchart which shows the detection method which concerns on 1st Embodiment. It is a figure which shows the detection apparatus which concerns on 2nd Embodiment. It is a figure which shows the detection apparatus which concerns on 3rd Embodiment. It is a figure which shows the detection apparatus which concerns on 4th Embodiment.

[First Embodiment]
The first embodiment will be described. FIG. 1 is a diagram showing a detection device according to the first embodiment. The detection device 1 (detection system) is, for example, a motion capture device, a motion detection system, a motion support system, or the like. Further, the detection device 1 is used for posture analysis, three-dimensional modeling, and the like. In these cases, the detection device 1 detects the object M moving in one direction or a plurality of directions in a predetermined time range. The detection device 1 may detect a linearly moving object M, or may detect a meandering and moving object M. The movement path (eg, locus) of the object M when the detection device 1 detects the object M may include a straight line, a curved line, or a straight line and a curved line. The detection device 1 is arranged on the first portion M1 of the object M arranged on the first side (eg, left side) of the side of the moving direction MD of the object M, and on the side opposite to the first side (eg, right side). It is discriminated from the second part M2 of the object M.

The object M is movable in the target area AR (eg, the detection area of the detection device 1, the field of view) to be detected by the detection device 1. The object M includes, for example, a human body or a non-human animal, a humanoid or non-human animal type robot, or a non-animal type robot. In the following description, it is assumed that the object M is a human body, and the object M is appropriately referred to as a human body M. The object M (in this case, the human body M) is, for example, an object whose posture and shape change with one or both of its movements. For example, when the human body M moves (eg, walking, running, exercising, moving), the first part M1 (eg, left foot) and the second part M2 (eg, right foot) move alternately in the moving direction. Therefore, the shape (eg, posture) of at least a part of the human body M changes. In the case of the human body M, the detection device 1 is arranged on the first side (eg, left side) with the moving direction MD of the human body M or the median line of the human body M (eg, the left and right center lines on the body surface) as a boundary. The first portion M1 and the second portion M2 arranged on the side opposite to the first side (eg, the right side) can be distinguished.

The moving human body M is, for example, sports such as fencing, baseball, soccer, golf, kendo, American football, ice hockey, or gymnastics, running, exercise, yoga, body building, walking or posing such as fashion shows, games, people. It is a human body that operates in authentication or work. The object M may include a moving human body and an object (eg, clothing, attachment, exercise equipment, armor) attached to the moving human body.

In the following description, the XYZ Cartesian coordinate system shown in FIG. 1 and the like will be referred to. In this XYZ Cartesian coordinate system, the X direction is the moving direction of the object M (eg, the human body M), the Y direction is the vertical direction, and the Z direction intersects the moving direction of the object M (human body M) and the vertical direction. (Example, orthogonal) Crossing direction (eg, orthogonal direction). When the object M is a human body, the Z direction includes the direction of the shoulder width of the human body M, the first part M1 is at least a part of the left half of the human body M, and the second part M2 is at least the right half of the human body M. It is a part. Further, in each of the X, Y, and Z directions, the side pointed by the arrow is referred to as a + side (eg, + Z side), and the opposite side is referred to as a − side (eg, −Z side). For example, the first side is the −Z side, and the second side is the + Z side.

The detection device 1 includes a position detection unit 2 and a processing device 3. The position detection unit 2 detects the position information of the moving human body M. The position detection unit 2 detects point cloud data including the three-dimensional coordinates of each point on the surface of the human body M as the position information of the human body M. In the present embodiment, the moving direction of the human body M is predetermined with respect to the target region AR (eg, the field of view of the position detection unit 2). The vertical direction and the moving direction of the human body M in the target region AR are given to the detection device 1 as known information. In the detection result (eg, point cloud data) of the position detection unit 2, each direction of X, Y, and Z is set based on the above-mentioned known information. The position detection unit 2 includes a detection unit 4 and a point cloud data generation unit 5.

The detection unit 4 is, for example, at least a part of a portable device (portable device). The detection unit 4 may be at least a part of a stationary device. The detection unit 4 may be provided inside the processing device 3. Further, the point cloud data generation unit 5 may be provided in a device outside the processing device 3. For example, the position detection unit 2 is an external device of the processing device 3, and may include the detection unit 4 and the point cloud data generation unit 5. A part or the whole of the detection device 1 may be a portable device (eg, an information terminal, a smartphone, a tablet, a camera-equipped mobile phone, a wearable terminal). Further, a part or the whole of the detection device 1 may be a stationary device (eg, a fixed point camera).

FIG. 2 is a diagram showing a detection unit according to the first embodiment. The detection unit 4 detects the depth as the position information of the human body M. The detection unit 4 includes, for example, a depth sensor (eg, a depth camera). The detection unit 4 detects the depth (eg, position information, distance, depth, depth) from a predetermined point to each point on the surface of the object arranged in the target area AR. The predetermined points are, for example, points at positions that serve as reference points for detection by the detection unit 4 (eg, a viewpoint, a point at the detection source, a point representing the position of the detection unit 4, and a position of a pixel of the image sensor 8 described later). ..

The detection unit 4 includes an irradiation unit 6, an optical system 7, and an image pickup device 8. The irradiation unit 6 irradiates (eg, projects) light La (eg, pattern light, irradiation light) on the target area AR (space, detection area). The optical system 7 includes, for example, an imaging optical system (imaging optical system). The image sensor 8 includes, for example, a CMOS image sensor or a CCD image sensor. The image pickup device 8 has a plurality of pixels arranged two-dimensionally. The image sensor 8 images the target region AR via the optical system 7. The image sensor 8 detects light Lb (infrared light, return light) emitted from an object in the target region AR by irradiation with light La.

The detection unit 4 is based on, for example, a pattern of light La emitted from the irradiation unit 6 (eg, intensity distribution) and a pattern of light Lb detected by the image sensor 8 (eg, intensity distribution, captured image). , The depth from the point on the target region AR corresponding to each pixel of the image sensor 8 to each pixel of the image sensor 8 is detected. As the detection result, the detection unit 4 outputs a depth map (eg, depth image, depth information, distance information) showing the depth distribution in the target area AR to the processing device 3 (see FIG. 1). The detection unit 4 outputs a depth map to the processing device 3 as the position information of the human body M.

The detection unit 4 may be a device that detects the depth by the TOF (time of flight) method. The detection unit 4 may be a device that detects the depth by a method other than the TOF method. The detection unit 4 may include, for example, a laser scanner (eg, a laser rangefinder) and detect the depth by laser scanning. The detection unit 4 may include, for example, a phase difference sensor and detect the depth by the phase difference method. The detection unit 4 may detect the depth by, for example, a DFD (depth from defocus) method.

The detection unit 4 may irradiate the object M with light other than infrared light (eg, visible light) and detect the light emitted from the object M (eg, visible light). The detection unit 4 may include, for example, a stereo camera, and may detect (eg, image) an object M from a plurality of viewpoints. The detection unit 4 may detect the depth by triangulation using the captured images obtained by capturing the object M from a plurality of viewpoints. The detection unit 4 may detect the depth by a method other than the optical method (eg, scanning by ultrasonic waves).

Returning to the description of FIG. 1, the point cloud data generation unit 5 generates point cloud data as position information of the human body M based on the depth detected by the detection unit 4. In FIG. 1, the detection unit 4 is provided outside the processing device 3, and the point cloud data generation unit 5 is provided inside the processing device 3. The processing device 3 is communicably connected to the detection unit 4. The detection unit 4 outputs the detection result to the processing device 3. The processing device 3 processes the detection result output from the detection unit 4.

The processing device 3 includes a point cloud data generation unit 5, a reference calculation unit 11, a partial discrimination unit 12, a posture estimation unit 13, and a storage unit 14. The storage unit 14 is, for example, a non-volatile memory, a hard disk (HDD), a solid state drive (SSD), or the like. The storage unit 14 stores the original data processed by the processing device 3 and the data processed by the processing device 3 (for example, the data generated by the processing device 3). The storage unit 14 stores the depth map output from the detection unit 4 as the original data of the point cloud data.

The point cloud data generation unit 5 executes a point cloud process that generates point cloud data as position information of the human body M. The point cloud data includes the three-dimensional coordinates of a plurality of points on the human body M in the target area AR. The point cloud data may include three-dimensional coordinates of a plurality of points on an object (eg, wall, floor) around the human body M. The point cloud data generation unit 5 calculates the point cloud data of the human body M based on the position information (eg, depth) detected by the detection unit 4. The point cloud data generation unit 5 reads out the detection result (eg, depth map) of the detection unit 4 stored in the storage unit 14 and calculates the point cloud data. The point cloud data generation unit 5 may generate point cloud data as model information (eg, shape information) of an object M (eg, human body M).

FIG. 3 is a diagram showing the processing of the point cloud data generation unit according to the first embodiment. Reference numeral D1 is a depth map (eg, depth image) corresponding to the detection result of the detection unit 4. The depth map D1 is information (example, image) representing the spatial distribution of the measured value of the depth by the detection unit 4. The depth map D1 is a grayscale image in which the depth at each point of the target area AR is represented by a gradation value. In the depth map D1, a portion having a relatively high gradation value (eg, a white portion, a bright portion) is a portion having a relatively small depth (eg, a portion relatively close to the position detection unit 2). In the depth map D1, a portion having a relatively low gradation value (eg, a black portion, a dark portion) is a portion having a relatively large depth (eg, a portion relatively far from the position detection unit 2).

The point cloud data generation unit 5 (see FIG. 1) reads the data of the depth map D1 from the storage unit 14, and sets each pixel based on the gradation value (eg, the measured value of the depth) of each pixel of the depth map D1. The three-dimensional coordinates of the corresponding points in the real space are calculated, and the point cloud data D2 is generated. In the following description, the three-dimensional coordinates of one point are appropriately referred to as point data. The point cloud data D2 is data in which a plurality of point data are set as a set.

The processing device 3 (eg, the point cloud data generation unit 5) extracts the point cloud data D2 of the human body M by performing segmentation, pattern recognition, or the like on the entire point cloud data of the target area AR, for example. The point cloud data generation unit 5 executes an extraction process (eg, segmentation) for extracting the position information of a part of the object from the position information of the object arranged in the target area AR. Here, it is assumed that the object arranged in the target area AR includes an object M (eg, a human body) and an object around the object M (eg, a floor, a wall, a background object). The point cloud data generation unit 5 extracts (eg, segmentation, separation) the position information of the object M (eg, the human body M) from the position information detected by the position detection unit 2.

Prior to the above extraction process, the position detection unit 2 performs the position detection unit 2 in each of the first state in which the object M is not arranged in the target area AR and the second state in which the object M is arranged in the target area AR. The position information of the object in the target area AR is detected. In the extraction process, the point cloud data generation unit 5 calculates the difference between the detection result of the position detection unit 2 in the first state and the detection result of the position detection unit 2 in the second state, thereby calculating the difference between the object M (eg, the object M). The position information of the human body M) is extracted.

In the first state, the object M (eg, human body M) may be arranged in the target area AR, and the position of the object M (eg, human body M) in the target area AR may be different from the second state. For example, the first state is a state in which an object M (eg, a human body M) is arranged at a first position in the target area AR, and the second state is an object at a second position different from the first position in the target area. It may be in a state where M (eg, human body M) is arranged. The above extraction process may be executed by a processing unit other than the point cloud data generation unit 5. This processing unit may be provided in the processing device 3 or may be provided in a device outside the processing device 3. The above extraction process does not have to be executed.

Further, the processing device 3 may remove noise from the detection result of the position detection unit 2. The noise removing process for removing noise includes, for example, a process for removing spatial noise from the depth map generated by the position detection unit 2. For example, the processing device 3 determines the depth in the first region when the difference in depth between the adjacent first region (eg, first pixel) and the second region (eg, second pixel) exceeds the threshold value in the depth map. Alternatively, it is determined that the depth in the second region is noise. When the processing device 3 determines that the depth in the first region is noise, the processing device 3 performs interpolation using the depth of the region (eg, third pixel, fourth pixel) around the first region to obtain the first region. Spatial noise is removed by estimating the depth and updating (replacing, correcting) the depth of the first region with the estimated depth. The point cloud data generation unit 5 may generate point cloud data based on the depth map from which spatial noise has been removed.

Further, the noise removing process for removing noise includes, for example, a process for removing temporal noise (eg, noise that changes with time) from the depth map generated by the position detection unit 2. For example, the position detection unit 2 repeats detection at a predetermined sampling frequency, and the processing device 3 compares the detection results (eg, depth map) of the position detection units 2 having different detection timings. For example, the processing device 3 calculates the amount of change in depth (eg, the amount of time change) in each pixel from the depth map in the first frame and the depth map in the second frame following the first frame. When the amount of change in depth in each pixel satisfies a predetermined condition, the processing device 3 determines that the depth in the first frame or the depth in the second frame of this pixel is noise. When the processing device 3 determines that the depth in the second frame is noise, the processing device 3 determines the depth of the second frame by interpolation using the depth in the first frame and the depth in the third frame following the second frame. presume. The processing device 3 removes temporal noise by updating (replacing, correcting) the depth of the pixel corresponding to the noise with the estimated depth. The point cloud data generation unit 5 may generate point cloud data based on the depth map from which temporal noise has been removed.

The noise removal process may not include a process for removing spatial noise or a process for removing temporal noise. Further, the noise removal process may be executed in a portion other than the processing device 3 (eg, the position detection unit 2). Further, the detection device 1 does not have to execute the noise removal process.

Returning to the description of FIG. 1, the processing device 3 sets the reference position BP associated with the shape of the human body M based on the pattern in which the cross-sectional shape of the human body M changes in a predetermined direction, and the human body with respect to the set reference. The left half body part and the right half body part of M are discriminated from each other. The reference calculation unit 11 sets a direction (eg, crossing direction, Z direction) different from both the moving direction (in this case, the X direction) and the vertical direction (in this case, the Y direction) in a predetermined direction. In the reference calculation unit 11, the cross-sectional shape (eg, contour) of the human body M on a plane orthogonal to a predetermined direction (eg, a plane parallel to the XY plane or the XY plane) changes in a predetermined direction (eg, a pattern of the human body M). The reference position BP is set by using the cross-sectional feature).

The reference calculation unit 11 uses, for example, a cross-sectional feature of the human body M in a side view when the human body M is viewed from the side with respect to the moving direction (for example, a pattern in which the contour of the human body M in an XY plane cross section changes in the Z direction). Then, the characteristic parts of the human body (eg, arms, shoulders, head) are identified, and the reference position BP is set. The cross-sectional feature is specified by the amount of change in the position information of the point on the surface of the human body M. The point position information (point data in this case) is, for example, information in which the coordinates in the X direction (X coordinate), the coordinates in the Y direction (Y coordinate), and the coordinates in the Z direction (Z coordinate) are set. .. The amount of change in position information is the amount of change in the Y coordinate in the X direction (eg, dy / dx), the amount of change in the Z coordinate in the X direction (eg, dz / dx), and the amount of change in the X coordinate in the Y direction (eg, dz / dx). For example, dx / dy), the amount of change in the Z coordinate in the Y direction (eg, dz / dy), the amount of change in the X coordinate in the Z direction (eg, dx / dz), and the amount of change in the Y coordinate in the Z direction. It is represented by at least one of (eg, dy / dz). The cross-sectional features will be described with reference to FIGS. 4 (A) to (C), FIGS. 5 (A) and (B).

The reference calculation unit 11 executes a reference calculation process for calculating the reference position BP. The reference calculation unit 11 determines that the amount of change in position information is a predetermined condition in the intersection direction (eg, Z direction) where the moving direction of the human body M (in this case, the X direction) and the vertical direction (in this case, the Y direction) intersect. The position that satisfies the condition is calculated as the reference position BP. The reference calculation unit 11 executes the reference calculation process described later based on the detection result (eg, one frame depth image) in which the position detection unit 2 detects the human body M at an arbitrary timing.

The reference position BP includes a first reference position BP1 which will be described later in FIG. 4 (B) and a second reference position BP2 which will be described later in FIG. 5 (B). The reference position BP is a position used as a reference in the partial discrimination process for discriminating between the first portion M1 and the second portion M2. The amount of change in the position information includes, for example, the amount of change in coordinates (eg, difference) between a plurality of points representing the surface of the human body M at an arbitrary timing (time). The amount of change in the position information may be the slope of a curve or straight line (eg, differential coefficient) representing the surface of the human body M (eg, outer shape, contour, silhouette).

Next, the reference calculation process will be described. The reference calculation unit 11 estimates the position of a structure (eg, head, body) in which the number of structures included in the object M (eg, human body M) is one as the reference position BP. The reference calculation unit 11 estimates the position of the portion including the center of the human body M (eg, center line CL) in the crossing direction (eg, Z direction) as the reference position BP. The reference calculation unit 11 calculates the reference position based on the point cloud data. The reference calculation unit 11 reads the point cloud data of the human body M generated by the point cloud data generation unit 5 from the storage unit 14, and executes the reference calculation process.

4 (A) to 4 (C), 5 (A), and 5 (B) are diagrams showing the processing of the reference calculation unit according to the first embodiment. The reference calculation unit 11 executes the first reference calculation process for calculating the first reference position BP1 (see FIG. 4B) as the reference position BP. In the present embodiment, the first reference position BP1 is the first reference position BP1a on the first side (eg, −Z side, left half of the body) of the object M and the second side (eg, + Z side, right half of the body) of the object M. ) Includes the first reference position BP1b. Further, the reference calculation unit 11 executes a second reference calculation process for calculating the second reference position BP2 (see FIG. 5B) based on the first reference position BP1 as the reference position BP.

First, the first standard calculation process will be described. The reference calculation unit 11 determines the position of a predetermined part of the human body M (eg, between the base of the neck, between the neck and the torso, between the shoulder and the head, etc.) based on the characteristics of the change in the position information of the human body M. Is estimated as the first reference position BP1a. The reference calculation unit 11 calculates a position that satisfies the condition that the change amount of the coordinates of the points included in the point cloud data D2 is equal to or more than the threshold value as a predetermined condition as the first reference position BP1.

Here, in the processing of the reference calculation unit 11, it is assumed that the object assumed to be detected is a walking human body, and the shape (eg, posture) of the human body at an arbitrary timing is assumed. The position of the surface of the human body changes remarkably from the arm (eg, elbow) to the shoulder (eg, the amount of change in position information exceeds the threshold value). In the human body, the change in the surface position is gradual between the shoulder and the neck (eg, the amount of change in the position information is less than the threshold value). However, in the human body, the position of the surface of the human body changes remarkably from the shoulder to the head as compared with the shoulder portion (for example, the amount of change in the position information becomes equal to or more than the threshold value). The reference calculation unit 11 calculates a position that satisfies the condition that the amount of change in the position information (eg, the Y coordinate of the point data) is equal to or greater than the threshold value as the first reference position BP1.

The arm (arm portion) includes, for example, the scapulohumeral joint, and is a portion from the scapulohumeral joint to the fingertip. The shoulder (shoulder) is, for example, a portion between the scapulohumeral joint of the left arm and the scapulohumeral joint of the right arm and on the lower body side of the 7th cervical vertebra (base of the neck). In addition, the above-mentioned head includes, for example, a portion surrounded by the skull, a portion on the surface side of the skull, and a neck. The head includes, for example, the 7th cervical vertebra and the parietal region, and is the portion from the 7th cervical vertebra to the parietal region.

The reference calculation unit 11 divides the region in the Z direction, and calculates a candidate for the first reference position BP1 for each divided region. In FIG. 4A, reference numeral DA _i is a divided region (divided region, partial region). The subscript i is a number assigned to each divided area, and is, for example, an integer of 0 or more (i = 0, 1, 2, ...). Further, Z _i is the coordinates in the Z direction representing the region DA _i (eg, the coordinates of the center _{of the region DA i in the Z direction).} Reference calculation unit 11, the point group data D2 in the region DA _i as a set of points in a 2D region (e.g., ignoring the value of the Z-coordinate), executes a first reference calculation process. The reference calculation unit 11 executes the first reference calculation process, assuming that the Z coordinate of the point data included _{in the area DA i is Z i.}

The reference calculation unit 11 calculates the maximum value of the coordinates in the vertical direction (Y direction) at each coordinate in the intersection direction (eg, Z direction) for the points included in the point cloud data D2. The reference calculation unit 11 calculates the maximum value of the Y coordinate of the point data for each of _{the divided regions DA i.} Ymax in Figure 4 (A) _{(Z i)} is the maximum value of the Y coordinate of the point data in the region DA _i. Reference calculation unit 11, Ymax the _{(Z i),} the maximum value of the Y coordinate of the point data at each coordinate _{(Z i).} Reference calculation unit 11 repeats the process of calculating the while changing the _i Ymax _(Z i), for each i = 0, 1, 2 · · · calculating the Ymax _{(Z i).}

FIG. 4B shows a plot _{of Ymax (Z i} ) with respect to the coordinates (Z coordinates) in the crossing direction. Here, the amount of change in the position information with respect to the coordinates (Z coordinates) in the crossing direction is represented _{by ΔYmax (Z i).} ΔYmax (Z _i ) is represented by, for example, the following equation (1).
ΔYmax (Z _i ) = Ymax (Z _{i + 1} ) -Ymax (Z _i ) ... Equation (1)

Further, showed a plot of the absolute value of ΔYmax with respect to the coordinate in the transverse direction (Z-coordinate) _{(Z i)} FIG. 4 (C). In FIGS. 4B and 4C, the reference numeral Q1 is a portion corresponding to the arm of the human body M, the portion corresponding to the left arm is represented by the reference numeral Q1a, and the portion corresponding to the right arm is represented by the reference numeral Q1b. .. Further, the reference numeral Q2 is a portion corresponding to the shoulder of the human body M, the portion corresponding to the left shoulder is represented by the reference numeral Q2a, and the portion corresponding to the right shoulder is represented by the reference numeral Q2b. Further, the reference numeral Q3 is a portion corresponding to the head of the human body M.

_{The absolute value of ΔYmax (Z i} ), which is the amount of change in position information (| ΔYmax (Z _i ) |), is large in the arm portion Q1 and smaller in the shoulder portion Q2 than in the arm portion Q1. Further, | ΔYmax (Z _i ) | sharply increases from the shoulder portion Q2 to the head portion Q3. The reference calculation unit 11 calculates the reference position BP (eg, the first reference position BP1a) based on the characteristics (eg, change pattern) of the amount of change in the position information. In the reference calculation process, the reference calculation unit 11 sets a portion where | ΔYmax (Z _i ) | is equal to or less than the first threshold value V1 as a first feature portion (eg, left shoulder portion Q2a, right shoulder portion Q2b) ( Example, identification, identification, determination).

First, a process of calculating the first reference position BP1a on the first side (eg, -Z side, left half body side) of the human body M will be described. The reference calculation unit 11 is in the order of crossing direction (eg, Z direction) from one side (eg, first side, −Z side) to the other side (eg, second side, + Z side). | ΔYmax (Z _i ) | is compared with the first threshold value V1. The reference calculation unit 11 determines the _{position at which the magnitude relationship between | ΔYmax (Z i} ) | and the first threshold value V1 is switched between the left arm portion Q1a and the left shoulder portion Q2a (eg, arm end, shoulder end). Set as (eg, identify, identify). For example, the reference calculation unit 11 sets _{the position of Z i in} the left arm portion Q1 _{when | Ymax (Z i} ) | is larger than the first threshold value V1 and ΔYmax (Z _{i + 1} ) is equal to or less than the first threshold value V1. It is set at the end on the + Z side, and _{the position of Z i + 1} is set at the end on the −Z side in the left shoulder portion Q2.

Further, in the reference calculation unit 11, | Ymax (Z _i ) | The portion having the second threshold value V2 or more is defined as the second characteristic portion of the human body M (eg, the head portion Q3). For example, the reference calculation unit 11, a first feature portion above the other side (for example, + Z side) from the position set as the end of the -Z side of (e.g., part Q2a left shoulder) in the order toward, | [delta] Ymax _{(Z i} ) | And the second threshold value V2 are compared. The reference calculation unit 11 determines the _{position where the magnitude relationship between | ΔYmax (Z i} ) | and the second threshold value V2 is switched between the first feature portion (eg, left shoulder portion Q2a) and the second feature portion (eg, head portion). It is set (eg, identification, identification, determination) as a boundary with the part Q3) (eg, the base end of the left shoulder, the -Z side end of the head, the neck, the base of the neck). For example, when ΔYmax (Z _i ) is smaller than the second threshold value V2 and ΔYmax (Z _{i + 1} ) is equal to or higher than the second threshold value V2 _{, the reference calculation unit 11 sets the position of Z i} as the first characteristic portion (on the left shoulder). It is set at the + Z side end in the portion Q2), and _{the position of Zi + 1} is set at the −Z side end in the second feature portion (head portion Q3).

Further, the reference calculation unit 11 calculates the first reference position BP1 corresponding to the head of the human body as the reference position based on the amount of change in the position information from the arm to the head of the human body M. The reference calculation unit 11 calculates a position representing the boundary between the first feature portion (eg, the left shoulder portion Q2a) and the second feature portion (head portion Q3) as the first reference position BP1. For example, the reference calculation unit 11 calculates the position of the + Z side end in the first feature portion (left shoulder portion Q2a) as the first reference position BP1a.

Next, the reference calculation unit 11 calculates the first reference position BP1b in the same manner as the first reference position BP1a. In the process of calculating the first reference position BP1b, the reference calculation unit 11 describes the first feature in the order from the other side (eg, + Z side) to the opposite side (eg, −Z side) with respect to the human body M. Set the part and the second feature part. Then, the reference calculation unit 11 calculates the position of the end on the −Z side in the first feature portion (right shoulder portion Q2b) as the first reference position BP1b. The reference calculation unit 11 stores the calculated first reference position BP1 (eg, three-dimensional coordinate value) in the storage unit 14 of FIG.

The reference calculation unit 11 sets the position of the end on the −Z side in the second feature portion (head portion Q3) as the first reference position BP1a, and sets the position on the + Z side in the second feature portion (head portion Q3). The position of the end may be set as the first reference position BP1b. Further, the reference calculation unit 11 is between the position of the + Z side end in the first feature portion (left shoulder portion Q2a) and the position of the −Z side end in the second feature portion (head portion Q3). The position (eg, center) may be the first reference position BP1a. In this case, the reference calculation unit 11 has the position of the end on the −Z side in the first feature portion (part Q2b of the right shoulder) and the position of the end on the + Z side in the second feature portion (part Q3 of the head). The position between them (eg, the center) may be set as the first reference position BP1b. Further, the reference calculation unit 11 may calculate only one of the first reference position BP1a and the first reference position BP1b as the first reference position BP1.

Next, the second reference calculation process will be described with reference to FIGS. 5 (A) and 5 (B). The reference calculation unit 11 sets the reference position BP from a predetermined region AR2 (eg, a region in the Z direction corresponding to the head portion Q3) set for the first reference position BP1 shown in FIG. 4 (B). Select (eg, set, determine) the second reference position BP2. As described with reference to FIGS. 4 (B) and 4 (C), the reference calculation unit 11 has a first reference position BP1a on the first side (eg, −Z side) of the object M and a second side (eg, −Z side). The first reference position BP1b on the + Z side) is calculated. The reference calculation unit 11 sets a predetermined region (eg, a region including the reference position) between the first reference position BP1a on the first side and the first reference position BP1b on the second side. It is assumed that the center line CL of the human body M in the Z direction passes through a region between the first reference position BP1a and the first reference position BP1b (the predetermined region, the head portion Q3). The reference calculation unit 11 calculates a position satisfying a predetermined condition as the second reference position BP2 in the region between the first reference position BP1a and the first reference position BP1b.

In the second reference calculation process, the reference calculation unit 11 divides the region in the Z direction in the same manner as in FIG. 4A, and calculates a candidate for the second reference position BP2 for each divided region. Reference calculation unit 11, the point group data D2 in the divided region DA _i as a set of points in a 2D region (e.g., ignoring the value of the Z-coordinate), executes the second reference calculation process.

In the second reference calculation process, the reference calculation unit 11 is in front of the moving direction (X direction) in _{each coordinate (eg, Z = Z i} ) of the intersection direction (eg, Z direction) with respect to the points included in the point cloud data. Calculate the minimum value of the coordinates with (+ X side) as positive. First, the reference calculation unit 11 extracts point data on the front side (front side in the traveling direction, + X side in the X direction) of the human body M. For example, the reference calculation unit 11 divides the area in the Y direction and calculates the maximum value of the X coordinate of the point data for each divided area.

In FIG. 5A, the reference numeral DB _j is a divided region (divided region, partial region). The subscript j is a number assigned to each divided area, and is, for example, an integer of 0 or more (j = 0, 1, 2, ...). Further, Y _j is the coordinates in the Y direction representing the region DB _j (for example, the coordinates of the center _{of the region DB j in the Y direction).} The reference calculation unit 11 _{executes the first reference calculation process by using the point cloud data D2 in the area DB j} as a set of point data in the one-dimensional area (eg, ignoring the value of the Y coordinate). The reference calculation unit 11 executes the second reference calculation process, assuming that the Y coordinate of the point data included _{in the area DB j is Y j.}

Xmax (Y _j ) in FIG. 5A is the maximum value of the X coordinate of the point data in the area DB _j. The reference calculation unit 11 sets Xmax (Y _j ) as the maximum value of the X coordinate of the point data at each coordinate (Y _j). Xmax (Y _j ) corresponds to the X coordinate of the point data on the front side in the traveling direction (X direction) in _{each coordinate (Y j} ) in the vertical direction (Y direction). Reference calculation unit 11 repeats the process of calculating the while changing the _j Xmax _(Y j), calculates the Xmax _{(Y j)} for each j = 0,1,2 ···.

Reference calculation unit 11, based on the distribution of the maximum values of the X coordinate of the point data for the Y-coordinate _{(Y j) (Xmax (Y} j)), calculates the minimum value of Xmax _{(Y j)} with respect to the Y coordinate. _{Xmin (Z i} ) in FIG. 5 (A) is the minimum value of _{Xmax (Y j} ) with respect to the Y coordinate in the region DA _i in which the Z coordinate is Z _i (Z = Z _{i in the figure).} It is the Y coordinate of the point data corresponding to Xmin (Z _i). The position of Xmax (Y _j ) is, for example, a position above the shoulder of the human body M and below the chin.

Reference calculation unit 11 repeats the process of calculating the while changing the _i Xmin _(Z i), for each i = 0, 1, 2 · · · calculating the Xmin _{(Z i).} The FIG. 5 (B), the exhibited plot represents the distribution of Xmin _{(Z i)} with respect to the Z coordinates. The reference calculation unit 11 selects the second reference position BP2 based on the amount of change in the _{minimum value (Xmin (Z i} )) with respect to the coordinates in the intersection direction (eg, Z direction). When the neck of the human body M is approximated by an elliptical pillar, the surface of the neck becomes convex toward the front in the traveling direction, and the position of the tip thereof corresponds to the position of the center line CL of the human body M. The reference calculation unit 11 calculates the Z coordinate (Zc in FIG. 5B) at which _{Xmin (Z i) with respect to the Z coordinate is maximum (or maximum).}

The reference calculation unit 11 sets Xmin (Zc) as the X coordinate of the second reference position BP2. Further, the reference calculation unit 11 sets the Y coordinate (Yk in FIG. 5A) corresponding to Xmin (Zc) as the Y coordinate of the second reference position BP2. Further, the reference calculation unit 11 sets Zc as the Z coordinate of the second reference position BP2. As described above, the reference calculation unit 11 according to the present embodiment _{treats the point cloud data for each divided region DA i} as two-dimensional data and executes the reference calculation process, so that the processing load can be reduced, for example. be. The reference calculation unit 11 may treat the point cloud data as three-dimensional data and execute the reference calculation process.

In this way, the reference calculation unit 11 according to the present embodiment calculates the reference position BP (eg, the second reference position BP2) using the external features on the front side of the human body M. Since the shape of a general human body changes remarkably from the chin to the throat and the chest, the reference calculation unit 11 can calculate the second reference position BP2 with high accuracy. The reference calculation unit 11 may calculate the reference position BP using the external features of the rear surface side (eg, the rear side in the traveling direction, the back side) of the human body M.

Returning to the description of FIG. 1, the reference calculation unit 11 stores the calculated reference position BP in the storage unit 14. The partial discrimination unit 12 executes the partial discrimination process. The partial discrimination unit 12 discriminates between the first part M1 (eg, at least a part of the left half of the body) and the second part M2 (eg, at least a part of the right half of the body) of the object M (human body M) in the partial discrimination process. (Example, judgment, identification, identification). The first portion M1 is a part of the object M arranged on the first side (eg, −Z side) with respect to the reference plane BF described later. The second portion M2 is a part of the object M arranged on the second side (eg, + Z side) opposite to the first side with respect to the reference plane BF. The reference plane BF is a plane including a moving direction (X direction) and a vertical direction (Y direction). The reference plane BF is, for example, a plane that is parallel to the moving direction (X direction) and the vertical direction (Y direction) and includes the reference position BP (second reference position BP2).

The partial discrimination unit 12 executes the partial discrimination process based on the reference position BP calculated by the reference calculation unit 11. The partial discrimination unit 12 sets the reference surface BF based on the reference position BP calculated by the reference calculation unit 11. The partial discrimination unit 12 reads the reference position BP calculated by the reference calculation unit 11 from the storage unit 14 and sets the reference surface BF. The partial discrimination unit 12 sets a surface parallel to the X direction and the Y direction and including the second reference position BP2 as the reference surface BF.

Then, the partial discrimination unit 12 executes the partial discrimination process based on the point cloud data generated by the point cloud data generation unit 5. The reference calculation unit 11 reads the point cloud data generated by the point cloud data generation unit 5 from the storage unit 14, and the portion represented by one or more point data included in the point cloud data is on the first side with respect to the reference surface BF. It is determined whether it is arranged on the (eg, −Z side) or the second side (eg, + Z side).

In the present embodiment, the second reference position BP2 is a position estimated by the reference calculation unit 11 as the position of a point on the center line CL of the human body M, and the partial determination unit 12 is the second reference position BP2 as the reference position BP. Is used to execute the partial discrimination process. The partial discrimination unit 12 discriminates a portion arranged on the first side (eg, −Z side) with respect to the second reference position BP2 as the first portion M1. Further, the partial discrimination unit 12 discriminates a portion arranged on the second side (eg, + Z side) with respect to the second reference position BP2 as the second portion M2.

FIG. 6 is a diagram showing the processing of the partial discrimination unit according to the first embodiment. In FIG. 6, the reference numeral QX is a portion (example, point, region) to be subjected to the partial discrimination process. In the following description, the target portion of the partial discrimination process is referred to as a discrimination target portion. The discrimination target portion QX is set in advance. For example, the discrimination target portion QX is set in a part of the human body M (example, hand, foot), and in the partial discrimination unit 12, the discrimination target portion QX (example, foot) is the left half body (example, left foot) and the right half body. Determine which of (eg, right foot) it belongs to. The discrimination target portion QX may be set in a plurality of characteristic portions (eg, whole body) of the human body M. In this case, the partial discrimination unit 12 may set each part of the human body M to the discrimination target part QX in order, and execute the partial discrimination process for each part of the human body M.

In the partial discrimination process, the partial discrimination unit 12 calculates the geodesic line SL connecting the reference position BP and the discrimination target portion QX of the object M (human body M). The geodesic SL is the shortest line connecting two points along the surface of the object M (human body M). The partial discrimination unit 12 calculates the geodesic SL based on the shape of the surface of the human body M obtained from the point cloud data. The partial discrimination unit 12 uses the second reference position BP2 as the reference position BP, and calculates the geodesic line SL connecting the second reference position BP2 and the discrimination target portion QX. When the discrimination target portion QX is an area, the partial discrimination unit 12 calculates a geodesic SL connecting a point selected from the discrimination target portion QX (eg, the center of the discrimination target portion QX) and the second reference position BP2. do.

The partial discrimination unit 12 discriminates between the first portion M1 and the second portion M2 based on the relative positions of the geodesic line SL and the reference plane BF. First, a case where the entire geodesic line SL is arranged on the first side (eg, −Z side) or the second side (eg, + Z side) with respect to the reference plane BF will be described. When the entire geodesic line SL is arranged on the first side (eg, −Z side) with respect to the reference plane BF, the partial determination unit 12 determines that the geodesic SL is on the first side (eg, −Z side) with respect to the reference plane BF. When it is arranged on the −Z side), the relative position between the geodesic SL and the reference plane BF is specified. Further, when the entire geodesic line SL is arranged on the second side (eg, + Z side) with respect to the reference plane BF, the partial determination unit 12 determines that the geodesic line SL is on the second side (eg, + Z side) with respect to the reference plane BF. , + Z side), the relative position between the geodesic SL and the reference plane BF is specified.

Next, a part of the geodesic line SL is arranged on the first side (eg, −Z side) with respect to the reference plane BF, and a part of the geodesic line SL is arranged on the second side (eg, + Z side) with respect to the reference plane BF. The case where it is arranged on the side) will be described. The partial discrimination unit 12 sets the feature amount (first feature amount) of the portion arranged on the first side (eg, -Z side) with respect to the reference plane BF on the geodesic SL and the reference plane BF on the geodesic SL. On the other hand, the relative position between the geodesic SL and the reference plane BF is specified based on the feature amount (second feature amount) of the portion arranged on the second side (eg, + Z side). The partial discrimination unit 12 specifies the relative position between the geodesic line SL and the reference plane BF based on the magnitude relationship or ratio between the first feature amount and the second feature amount.

The feature amount is, for example, the distance between a point on the geodesic SL and a reference plane BF. The partial discrimination unit 12 includes the distance (first feature amount) between the point on the geodesic SL arranged on the first side (eg, -Z side) with respect to the reference surface BF and the reference surface BF, and the reference surface BF. The relative position between the geodesic SL and the reference plane BF is specified based on the distance (second feature amount) between the point on the geodesic SL arranged on the second side (+ Z side) and the reference plane. ..

For example, the partial discrimination unit 12 sets each point and the reference plane for a plurality of points (first set of points) on the geodesic SL arranged on the first side (eg, −Z side) with respect to the reference plane SF. The average distance from the SF (first feature amount) is calculated. Further, the partial discrimination unit 12 includes each point and the reference plane SF for a plurality of points (second point set) on the geodesic SL arranged on the second side (eg, + Z side) with respect to the reference plane SF. Calculate the average of the distances from and (second feature amount). When the first feature amount is larger than the second feature amount, the partial discrimination unit 12 identifies that the geodesic line SL is a relative position arranged on the first side (eg, −Z side) with respect to the reference plane BF. do. Further, when the second feature amount is larger than the first feature amount, the partial discrimination unit 12 determines that the geodesic line SL is a relative position arranged on the second side (eg, + Z side) with respect to the reference plane BF. Identify. The partial discrimination unit 12 may specify the relative position between the geodesic SL and the reference plane BF by comparing the ratio of the first feature amount and the second feature amount with the threshold value.

The partial discrimination unit 12 selects a plurality of points on the geodesic SL when setting (generating) the first point set and the second point set. The partial discrimination unit 12 may select the plurality of points regularly (eg, at predetermined intervals) or irregularly (eg, randomly). Then, the partial discrimination unit 12 classifies each selected point into a first set of points when it is arranged on the first side (eg, −Z side) with respect to the reference plane SF, and the reference plane. When the points are arranged on the second side (eg, + Z side) with respect to the SF, this point is classified into the second point set, and the first point set and the second point set are set.

The partial discrimination unit 12 sets the first point set and the second point set so that the number of points belonging to the first point set and the number of points belonging to the second point set are the same. It may be set. In this case, the partial discrimination unit 12 may use the sum of the distances as the feature amount instead of the average value of the distances. For example, the partial discrimination unit 12 uses the sum of the distances related to the first point set as the first feature amount and the sum of the distances related to the second point set as the second feature amount, and uses the geodesic SL and the reference plane BF. You may specify the relative position with.

Further, the first feature amount may be the distance between the point on the geodesic line SL farthest from the reference plane BF to the first side (eg, −Z side) and the reference plane BF. The second feature amount may be the distance between the point on the geodesic line SL farthest from the reference plane BF to the second side (+ Z side) and the reference plane BF. When the first feature amount is larger than the second feature amount (long distance), the partial discrimination unit 12 is relative to the geodesic line SL being arranged on the first side (eg, −Z side) with respect to the reference plane BF. Identify the location. Further, in the partial discrimination unit 12, when the second feature amount is larger than the first feature amount (long distance), the geodesic line SL is arranged on the second side (eg, + Z side) with respect to the reference plane BF. Identify as a relative position. The partial discrimination unit 12 may specify the relative position between the geodesic SL and the reference plane BF by comparing the ratio of the first feature amount and the second feature amount with the threshold value.

Further, the first feature amount may be the length of a portion of the geodesic line SL arranged on the first side (eg, −Z side) with respect to the reference plane BF. The second feature amount may be the length of a portion of the geodesic line SL that is arranged on the second side (eg, + Z side) with respect to the reference plane BF. When the first feature amount is larger (or longer) than the second feature amount, the partial discrimination unit 12 is located at a relative position where the geodesic SL is arranged on the first side (eg, −Z side) with respect to the reference plane BF. Identify as. Further, in the partial discrimination unit 12, when the second feature amount is larger (or longer) than the first feature amount, the geodesic line SL is arranged on the second side (eg, + Z side) with respect to the reference plane BF. Identify the location. The partial discrimination unit 12 may specify the relative position between the geodesic SL and the reference plane BF by comparing the ratio of the first feature amount and the second feature amount with the threshold value.

Further, the partial discrimination unit 12 does not have to set (generate) the first point set and the second point set, and does not use the first feature amount and the second feature amount, and uses the geodesic SL. The relative position with respect to the reference plane BF may be specified. For example, the partial discrimination unit 12 selects a plurality of points on the geodesic SL regularly (eg, at predetermined intervals) or irregularly (eg, randomly). Then, the partial discrimination unit 12 calculates the distance from the reference plane BF for each selected point. The partial discrimination unit 12 represents the distance from the reference surface BF with a negative value when the selected points are arranged on one side (eg, the first side, −Z side) with respect to the reference surface BF. Further, the partial discrimination unit 12 represents the distance from the reference surface BF as a positive value when the selected points are arranged on the other side (eg, the second side, + Z side) with respect to the reference surface BF. .. The partial discrimination unit 12 calculates the sum of the distances between each point represented by a positive value, a negative value, or 0 and the reference plane BF for a plurality of points on the geodesic line SL. The partial discrimination unit 12 is at a relative position where the geodesic SL is arranged on one side (eg, the first side, −Z side) with respect to the reference plane BF when the sum of the calculated distances is a negative value. Identify as being. Further, the partial determination unit 12 is a relative position where the geodesic line SL is arranged on the other side (eg, the second side, + Z side) with respect to the reference plane BF when the sum of the calculated distances is a positive value. Identify as.

The partial determination unit 12 may specify the relative position between the geodesic line SL and the reference surface BF based on the relative position between the predetermined portion on the geodesic line SL and the reference surface BF. The above-mentioned predetermined portion may be the point farthest from the reference plane BF (farthest point) on the geodesic line SL. In the partial discrimination unit 12, when the farthest point is arranged on the first side (eg, −Z side) with respect to the reference plane BF, the geodesic line SL is on the first side (eg, −Z side) with respect to the reference plane BF. It is specified as a relative position arranged on the Z side). Further, in the partial discrimination unit 12, when the farthest point is arranged on the second side (eg, + Z side) with respect to the reference plane BF, the geodesic line SL is on the second side (eg, + Z side) with respect to the reference plane BF. It is specified that it is a relative position arranged on the + Z side). The predetermined portion may be a portion other than the farthest point, for example, a portion closer to the second reference position BP2 than the discrimination target portion QX on the geodesic line SL (for example, a predetermined length starting from the second reference position BP2). Part) may be. The partial discrimination unit 12 determines that the discrimination target portion QX belongs to the second portion M2 when a portion of the geodesic line SL having a predetermined length starting from the second reference position BP2 extends to the second side (+ Z side). You may discriminate. The partial discrimination unit 12 may specify the relative position between the geodesic line SL and the reference plane BF by combining the above methods.

In FIG. 6, the geodesic line SL extends from the second reference position BP2 to the + Z side, and reaches the discrimination target portion QX via the + Z side of the reference surface BF. In such a case, the partial discrimination unit 12 determines that the discrimination target portion QX belongs to the second portion M2 (eg, the right half of the body). Here, it is assumed that the left foot and the right foot intersect or line up on the same line. In this state, the right foot is placed at or at the same position as the left foot in the direction of the shoulder width of the human body. Therefore, it may be difficult for a conventional conventional device to automatically determine whether the detected foot is the right foot or the left foot. In a state where the left foot and the right foot intersect or line up on the center line of the human body, the detection device 1 according to the embodiment uses at least the relative position of the geodesic SL and the reference plane BF, for example. Since a part of the foot is arranged on the + Z side or the −Z side with respect to the reference plane BF, it is possible to automatically determine with high accuracy whether the detected foot is the left foot or the right foot.

Returning to the description of FIG. 1, as described above, the partial discrimination unit 12 executes the partial discrimination processing on at least a part of the object M (human body M) represented by the point cloud data, and stores the processing result in the storage unit 14. To memorize. For example, the partial discrimination unit 12 executes a partial discrimination process for each portion of the object M (human body M) represented by the point cloud data, and indicates whether this portion belongs to the first portion M1 or the second portion M2. The determination information (eg, flag, attribute information) is stored in the storage unit 14 as a processing result.

Then, the posture estimation unit 13 executes a posture estimation process for estimating the posture of the object M based on the first portion M1 and the second portion M2 determined by the partial discrimination unit 12. The partial discrimination unit 12 generates, for example, position information of a characteristic portion (example, feature portion, feature point) of the human body M. The characteristic part of the human body M is, for example, a part of the human body M that can be distinguished from other parts. The characteristic portion of the human body M includes, for example, at least one of the terminal parts (hands, toes, head), joints, or intermediate parts between the terminal parts and the joints or between the two joints.

The posture estimation unit 13 executes recognition processing (eg, pattern recognition, shape recognition, skeleton recognition) on the shape of the human body M obtained from the point cloud data, for example. The posture estimation unit 13 generates the position information of the above-mentioned feature portion by the recognition process. The position information of the feature portion includes, for example, the coordinates of a point representing the feature portion (eg, three-dimensional coordinates). The posture estimation unit 13 calculates the coordinates of the points representing the feature portions by the above recognition process. Then, the posture estimation unit 13 stores the information of the specified portion (eg, the coordinates of the point representing the feature portion) in the storage unit 14.

FIG. 7 is a diagram showing processing of the posture estimation unit according to the first embodiment. In FIG. 7, reference numerals Q11 to Q30 are characteristic portions of the human body M specified by the posture estimation unit 13. Reference numerals Q11 to Q15 are characteristic portions corresponding to the terminal portions, Q11 is the head, Q12 is the left toe, Q13 is the left hand, Q14 is the right toe, and Q15 is the right hand. The symbols Q16 to Q27 are characteristic parts corresponding to the joints, Q16 is the left ankle, Q17 is the left knee, Q18 is the left hip joint (base of the left foot), Q19 is the right ankle, Q20 is the right knee, and Q21 is. Right hip joint (base of right foot), Q22 is the left wrist, Q23 is the left elbow, Q24 is the left scapulohumeral joint, Q25 is the right wrist, Q26 is the right elbow, and Q27 is the right scapulohumeral joint. The symbols Q28 to Q30 are characteristic portions corresponding to the intermediate portion between the end portion and the joint or between the two joints, Q28 is the waist (center of the left and right hip joints), and Q29 is the neck (left and right). The center of the scapulohumeral joint) and Q30 are the back (the center of the waist Q28 and the neck Q29).

In the posture estimation unit 13 of the present embodiment, since the partial discrimination unit 12 can discriminate between the left half body part and the right half body part with high accuracy, the characteristic parts (eg, left ankle, right ankle) of the human body M can be identified. It can be specified with high accuracy. As a result of the above recognition process, the posture estimation unit 13 determines the position of each feature portion, the information of each feature portion (eg, attribute information, name (right knee, right hip joint)), and the connection relationship between the two feature portions. Posture information (eg, skeleton information, skeleton data) that is a set of the connection information shown is generated. The posture estimation unit 13 stores the posture information in the storage unit 14 of FIG.

The posture estimation unit 13 estimates the posture of the human body M based on the posture information. The posture estimation unit 13 has a relative position (eg, angle) between a first line connecting a pair of characteristic parts having a connection relationship and a second line connecting a pair of feature parts having a connection relationship with the first line. Based on, the posture of the part including the first line and the second line is estimated. Here, the first line is the right shin Q31 connecting the right ankle Q19 and the right knee Q20, and the second line is the right thigh Q32 (thigh) connecting the right knee Q20 and the right hip joint Q21. Suppose that The posture estimation unit 13 calculates the angle α formed by the right shin Q31 and the right thigh Q32. The posture estimation unit 13 determines that the right leg of the human body M including the right shin Q31 and the right thigh Q32 is in a bent posture when the angle α formed by the right shin Q31 and the right thigh Q32 is equal to or less than the threshold value. do. When the angle α formed by the right shin Q31 and the right thigh Q32 is larger than the threshold value, the posture estimation unit 13 determines that the right leg of the human body M including the right shin Q31 and the right thigh Q32 is in an extended posture. do.

The posture estimation unit 13 estimates the posture of a part or the whole of the human body M by, for example, estimating the posture of each feature portion preset by the user as described above. The posture estimation unit 13 may estimate the posture by referring to the posture definition information that defines the posture of the object (eg, the human body). The above-mentioned posture definition information is, for example, information that represents a type of posture and information that defines a relative position of each feature portion as a set. The information indicating the type of posture is, for example, a posture name such as a walking posture, a sitting position, or a yoga pose name. The information that defines the relative position of each feature portion is, for example, the range or threshold value of the angle α formed by the right shin Q31 and the right thigh Q32. The posture estimation unit 13 may estimate (identify) the posture of the human body M by collating the generated posture information with the posture definition information.

Next, the detection method according to the embodiment will be described based on the configuration of the detection device 1. FIG. 8 is a flowchart showing a detection method according to the first embodiment. Regarding the configuration of the detection device 1 and the processing by each part, FIGS. 1 to 7 are referred to as appropriate. In step S1, the position detection unit 2 detects the position information of each point on the surface of the object M. The position detection unit 2 detects the depth as the position information and generates point cloud data representing the position information as the detection result (see FIG. 3). The process of step S1 includes the process of step S2 and the process of step S3. In step S2, the detection unit 4 detects the depth from a predetermined point (viewpoint) to each point of the target area AR. The detection unit 4 stores a depth map (spatial distribution of depth) representing the detection result in the storage unit 14. In step S3, the point cloud data generation unit 5 generates point cloud data based on the depth detection result. The point cloud data generation unit 5 stores the generated point cloud data in the storage unit 14.

In step S4, the reference calculation unit 11 calculates a position where the amount of change in the position information satisfies a predetermined condition as the reference position BP. The reference calculation unit 11 calculates the first reference position BP1 and the second reference position BP2 as the reference position BP. The process of step S4 includes the process of step S5 and the process of step S6.

In step S5, the reference calculation unit 11 calculates the first reference position BP1 based on the amount of change in the coordinates (position information) in the vertical direction (Y direction) (see FIG. 4). The reference calculation unit 11 stores the calculated first reference position BP1 in the storage unit 14. In step S6, the reference calculation unit 11 calculates the second reference position BP2 based on the first reference position BP1 (see FIG. 5). The reference calculation unit 11 sets a predetermined area AR2 based on the first reference position BP1 calculated in step S5, and calculates a position in the predetermined area AR2 that satisfies a predetermined condition as the second reference position BP2. The reference calculation unit 11 stores the calculated second reference position BP2 in the storage unit 14.

Next, in step S7, the partial discrimination unit 12 has the first portion M1 on the first side (−Z side) and the second portion on the second side (+ Z side) with respect to the reference surface BF based on the reference position BP. Distinguish from M2. The partial discrimination unit 12 sets the reference plane BF based on the second reference position BP2 calculated in step S6. Further, the partial discrimination unit 12 calculates the geodesic line SL connecting the discrimination target portion QX of the partial discrimination process and the second reference position BP2.

The partial discrimination unit 12 determines whether the discrimination target portion QX belongs to the first portion M1 or the second portion M2 based on the relative position between the reference plane BF and the geodesic line SL. The partial discrimination unit 12 generates discrimination information indicating whether the discrimination target portion QX belongs to the first portion M1 or the second portion M2 as the processing result of the partial discrimination processing. The partial discrimination unit 12 stores the generated discrimination information in the storage unit 14.

In step S8, the posture estimation unit 13 estimates the posture of the object M. The posture estimation unit 13 executes a posture estimation process for estimating the posture of the object M (human body M) based on the processing result (eg, discrimination information) of the partial discrimination unit 12. The posture estimation unit 13 executes a recognition process on the shape of the human body M obtained from the point cloud data, and identifies a characteristic portion of the human body M (see FIG. 7). The posture estimation unit 13 estimates the posture of the human body M based on the relative positions of the specified plurality of feature portions.

As described above, in the detection device 1 according to the embodiment, the reference calculation unit 11 calculates the reference position BP based on the amount of change in the position information, and the partial determination unit 12 calculates the reference position BP based on the reference position BP on the first side of the object M. The first portion M1 on the (eg, −Z side, left side) and the second portion M2 on the second side (eg, + Z side, right side) are discriminated from each other. Therefore, the detection device 1 can discriminate between the first part M1 (eg, left foot) and the second part M2 (eg, right foot) with high accuracy. For example, the posture estimation unit 13 is an object M (human body M). The posture can be estimated with high accuracy.

Further, the reference calculation unit 11 according to the present embodiment estimates the position of the portion including the center of the object M in the crossing direction (eg, the Z direction) as the reference position BP. In this case, since the partial discrimination unit 12 executes the partial discrimination process based on the position of the portion including the center of the object M, the first portion M1 on the first side and the second portion M2 on the second side are highly accurate. Can be determined.

Next, a modified example will be described. The reference position BP may be deviated from the center of the object M in the Z direction. For example, it is assumed that the reference position BP is deviated from the center of the object M to the first side (+ Z side) by a predetermined amount of deviation. Further, it is assumed that the maximum distance between the geodesic SL (see FIG. 6) and the reference plane BF is large with respect to the amount of deviation between the reference position BP and the center of the object M. In this case, the partial discrimination unit 12 is based on the feature amount (eg, the ratio of the geodesic line SL to the whole, the reference surface BF) of the portion arranged on the first side (+ Z side) with respect to the reference surface BF on the geodesic line SL. On which side the geodesic SL is located with respect to the reference plane BF) may be determined based on (the maximum distance of). For example, in the partial discrimination unit 12, the discrimination target portion QX is arranged on the first side when the ratio of the portion arranged on the first side (+ Z side) with respect to the reference plane BF on the geodesic line SL is equal to or more than the threshold value. It may be determined that it is a part that has been removed.

The position detection unit 2 does not have to calculate the point cloud data as the position information. In this case, the reference calculation unit 11 may calculate the reference position BP by processing the depth detected by the position detection unit 2 (detection unit 4). Further, the reference calculation unit 11 may calculate the reference position BP based on the shape information (eg, polygon data) of the object M obtained from the depth detected by the position detection unit 2 (detection unit 4) (later). This will be described with reference to FIG. 10).

The reference calculation unit 11 does not have to set a predetermined area AR2. For example, the reference calculation unit 11 may set the center of the first reference position BP1a and the first reference position BP1b at the second reference position BP2. Further, the reference calculation unit 11 may calculate only one of the first reference position BP1a and the first reference position BP1b as the first reference position BP1. In this case, the reference calculation unit 11 may set the position shifted from the first reference position BP1 in the Z direction by a predetermined amount to the second reference position BP2. The predetermined amount may be set based on, for example, the size (scale) of the object M. For example, the predetermined amount may be set to a predetermined ratio (eg, 25%) to the size of the object M in the Z direction (eg, the distance between the left end of the left shoulder and the right end of the right shoulder). .. Further, the reference calculation unit 11 does not have to calculate the first reference position BP1. _{For example, the reference calculation unit 11 has a pattern in which the feature amount (eg, Xmin (Z i} )) on the front side or the back side of the object M changes in the Z direction as shown in FIG. 5 (A) (eg, FIG. 5 (B). ), The positions of X = Xmin (Zc), Y = Yk, and Z = Zc may be calculated as the reference position BP.

The detection device 1 does not have to include the posture estimation unit 13. The detection device 1 may calculate the feature amount (eg, dimensions, outer peripheral length, cross-sectional area) of the portion determined by the partial discrimination unit 12. Further, the detection device 1 may segment the object M by using the processing result of the partial discrimination process by the partial discrimination unit 12. The detection device 1 uses the processing result of the partial discrimination process by the partial discrimination unit 12 (for example, information for identifying whether the detected "foot" is the "left foot" or the "right foot") to model the object M. Information may be generated (described later in FIG. 10).

[Second Embodiment]
Next, the second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 9 is a diagram showing a detection device according to the second embodiment. In the first embodiment, the moving direction is predetermined with respect to the target region AR (eg, the field of view of the position detection unit 2), but in the present embodiment, the detection device 1 is an object M (eg, the field of view of the position detection unit 2). , The moving direction of the human body M) is detected. The detection device 1 includes a direction calculation unit 21 that calculates (derives, detects) the movement direction of the object M (human body M) based on the detection result of the position detection unit 2.

In the present embodiment, the detection unit 4 repeatedly detects the human body M, and the point cloud data generation unit 5 generates point cloud data of the human body M for each detection timing as position information. The direction calculation unit 21 calculates the moving direction of the human body M based on the time change of the position information of the human body M detected by the position detection unit 2. For example, the direction calculation unit 21 calculates the locus of the human body M (eg, the time history of a predetermined position of the human body M), and sets the direction along the locus as the moving direction of the human body M. The predetermined position is a position on the human body M determined by the default setting or the user's setting.

The direction calculation unit 21 calculates the centers of gravity of a plurality of points included in the point cloud data as predetermined positions of the human body M. The direction calculation unit 21 calculates the moving direction based on the calculated time change of the center of gravity. For example, the direction calculation unit 21 starts from the first predetermined position of the human body M obtained from the detection result of the detection unit 4 at the first time, and obtains from the detection result of the detection unit 4 at the second time after the first time. A vector having a second predetermined position of the human body M as an end point is calculated, and a direction vector (eg, a unit vector) parallel to this vector is used to determine the moving direction of the human body M at the first time.

The direction calculation unit 21 stores the calculated movement direction (eg, movement information) in the storage unit 14. The reference calculation unit 11 calculates the reference position BP based on the movement direction calculated by the direction calculation unit 21. For example, the reference calculation unit 11 reads the movement direction calculated by the direction calculation unit 21 from the storage unit 14, sets the movement direction to the X direction, and executes the reference calculation process. The partial discrimination unit 12 discriminates between the first portion M1 and the second portion M2 based on the movement direction calculated by the direction calculation unit 21. For example, the partial discrimination unit 12 sets a plane parallel to the vertical direction and the movement direction calculated by the direction calculation unit 21 and including the second reference position BP2 as the reference plane BF. Then, the partial discrimination unit 12 determines whether the discrimination target portion QX is arranged on the first side (example, −Z side) or the second side (example, + Z side) with respect to the set reference plane BF. Determine.

The predetermined position may be the position of the characteristic portion (eg, the head of the human body M) described in the first embodiment. Further, the predetermined position may be the position of a characteristic portion (eg, right foot, left foot) of the human body M derived based on the discrimination result of the partial discrimination unit 12. Further, the predetermined position may be a reference position BP calculated by the reference calculation unit 11 (eg, a second reference position BP2). For example, the direction calculation unit 21 may calculate a candidate for a moving direction based on a locus of a first predetermined position (eg, a center of gravity) set in advance. Then, the reference calculation unit 11 may calculate the candidate of the reference position BP (eg, the second reference position BP2) with the candidate of the moving direction calculated by the direction calculation unit 21 as the X direction. The direction calculation unit 21 uses the candidate of the second reference position BP2 calculated by the reference calculation unit 11 as the second predetermined position, and calculates the moving direction based on the locus of the second predetermined position (eg, recalculation). You may. The reference calculation unit 11 may calculate (recalculate) the reference position BP with the movement direction calculated using the second predetermined position as the X direction.

The direction calculation unit 21 may calculate the moving direction of the object M based on the position information of the object M acquired from the Global Positioning System (GPS). Further, an acceleration sensor may be provided on the object M, and the direction calculation unit 21 may calculate the moving direction of the object M based on the detection result of the acceleration sensor. For example, the human body M moves with a mobile terminal (eg, a smartphone) including one or both of a receiving unit that receives information from GPS and the acceleration sensor, and the direction calculation unit 21 moves from the mobile terminal to the human body. The position information (eg, GPS information, acceleration) of M may be acquired to calculate the moving direction of the object M.

The detection device 1 includes a vertical detection unit (eg, a sensor that detects the direction of gravitational acceleration) that detects the vertical direction, and sets the detection result detected by the vertical detection unit in the Y direction described above to perform reference calculation processing. And one or both of the partial discrimination process may be executed. The detection device 1 does not have to include the vertical detection unit. For example, the vertical detection unit is provided on the object M (carried by the human body M), and the detection device 1 is one of a reference calculation process and a partial discrimination process based on the vertical direction acquired from the vertical detection unit. You may do both. Further, the vertical detection unit may be provided separately from both the detection device 1 and the object M. For example, the vertical detection unit may be provided at a place where the object M is detected (eg, equipment in which the detection device 1 is installed).

[Third Embodiment]
Next, the third embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 10 is a diagram showing a detection device according to a third embodiment. The detection device 1 includes a model generation unit 22 that generates model information of an object M (in this case, a human body M). The model information is, for example, three-dimensional CG model data, and includes shape information of the object M. The model generation unit 22 calculates the model information of the object M based on the first portion M1 and the second portion M2 determined by the partial discrimination unit 12.

The model generation unit 22 according to the present embodiment includes a point cloud data generation unit 5 and a surface information generation unit 23. As described in the first embodiment, the point cloud data generation unit 5 generates point cloud data as shape information based on the depth of the object M detected by the detection unit 4. The surface information generation unit 23 executes surface processing for calculating surface information as shape information.

The surface information includes, for example, at least one of polygon data, vector data, and draw data. The surface information includes the coordinates of a plurality of points on the surface of the object and the connection information between the plurality of points. The connection information (eg, attribute information) includes, for example, information that associates the points at both ends of the line corresponding to the ridge line (eg, edge) on the surface of the object with each other. Further, the connection information includes, for example, information for associating a plurality of lines corresponding to the contour of the surface of the object with each other.

The surface information generation unit 23 estimates the surface between a point selected from a plurality of points included in the point cloud data and a point in the vicinity thereof in the surface processing. The surface information generation unit 23 segments the point cloud data using the processing result of the partial discrimination unit 12 when estimating the surface between the points and the points in the vicinity thereof. For example, the surface information generation unit 23 distinguishes the point cloud data of the left foot from the point cloud data of the right foot based on the processing result of the partial discrimination unit 12 when generating the surface information of the left foot of the human body M. In this case, the surface information generation unit 23 avoids estimating the surface straddling the left knee and the right knee, for example, when the left knee and the right knee are close to each other (eg, contact), and is high. Accurate surface information can be generated.

Further, the surface information generation unit 23 converts the point cloud data into polygon data having plane information between points in the surface processing. The surface information generation unit 23 converts the point cloud data into polygon data by, for example, an algorithm using the least squares method. This algorithm may be, for example, an application of an algorithm published in a point cloud processing library. The surface information generation unit 23 stores the calculated surface information in the storage unit 14.

The model information may include the texture information of the object M. The model generation unit 22 may generate the texture information of the surface defined by the three-dimensional point coordinates and the related information. The texture information includes, for example, at least one piece of information such as characters and figures on the surface of an object, a pattern, a texture, a pattern, information defining irregularities, a specific image, and a color (eg, chromatic color, achromatic color). The model generation unit 22 may store the generated texture information in the storage unit 14.

Further, the model information may include spatial information (eg, lighting conditions, light source information) of the image. The light source information includes the position of the light source that irradiates the object M with light (eg, illumination light), the direction in which the light is emitted from this light source to the object M (irradiation direction), and the wavelength of the light emitted from this light source. And information on at least one item of this type of light source. The model generation unit 22 may calculate the light source information by using, for example, a model assuming Lambertian reflection, a model including Albedo estimation, and the like. The model generation unit 22 may store the generated spatial information in the storage unit 14. The model generation unit 22 does not have to generate one or both of the texture information and the spatial information.

[Fourth Embodiment]
Next, the fourth embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 11 is a diagram showing a detection device according to a fourth embodiment. The detection device 1 (detection system) according to the present embodiment includes a model generation unit 22, a rendering processing unit 24 (rendering processing device, information processing device), an input device 25, and a display device 26.

The rendering processing unit 24 includes, for example, a graphics processing unit (GPU). The rendering processing unit 24 may have a mode in which the CPU and the memory execute each processing according to the image processing program. The rendering processing unit 24 performs at least one of drawing processing, texture mapping processing, and shading processing, for example.

In the drawing process, the rendering processing unit 24 can calculate, for example, an estimated image (eg, a reconstructed image) of the shape defined in the shape information of the model information viewed from an arbitrary viewpoint. In the following description, the shape indicated by the shape information is referred to as a model shape. The rendering processing unit 24 can reconstruct the model shape (example, estimated image) from the model information (example, shape information) by, for example, drawing processing. The rendering processing unit 24 stores, for example, the calculated estimated image data in the storage unit 14.

Further, in the texture mapping process, the rendering processing unit 24 can calculate, for example, an estimated image in which the image indicated by the texture information of the model information is pasted on the surface of the object on the estimated image. The rendering processing unit 24 can calculate an estimated image in which a texture different from the object is attached to the surface of the object on the estimated image.

In the shading process, the rendering processing unit 24 can calculate, for example, an estimated image in which a shadow formed by the light source indicated by the light source information of the model information is added to the object on the estimated image. Further, in the shading process, the rendering processing unit 24 can calculate, for example, an estimated image in which a shadow formed by an arbitrary light source is added to an object on the estimated image.

The input device 25 is used for inputting various information (eg, data, commands) to the processing device 3. The user can input various information to the processing device 3 by operating the input device 25. The input device 25 includes, for example, at least one of a keyboard, a mouse, a trackball, a touch panel, and a voice input device (eg, a microphone).

The display device 26 displays this image based on the image data output from the processing device 3. For example, the processing device 3 outputs the data of the estimated image generated by the rendering processing unit 24 to the display device 26. The display device 26 displays the estimated image based on the estimated image data output from the processing device 3. The display device 26 includes, for example, a liquid crystal display. The display device 26 and the input device 25 may be a touch panel or the like.

The detection device 1 does not have to include the input device 25. For example, the detection device 1 may be in a form in which various commands and information are input via communication. Further, the detection device 1 does not have to include the display device 26. For example, the detection device 1 may output the data of the estimated image generated by the rendering process to the display device via communication, and this display device may display the estimated image. The rendering processing unit 24 is provided in the processing device 3 in FIG. 11, but may be provided in a device outside the processing device 3. The external device may be a cloud computer that is communicably connected to the processing device 3.

In the above-described embodiment, the processing device 3 includes, for example, a computer system. The processing device 3 reads a processing program stored in the storage unit 14 and executes various processes according to the processing program. In this processing program, for example, in a computer, a reference position is set at a position where the amount of change in position information of each point on the surface of the object satisfies a predetermined condition in an intersection direction where the moving direction and the vertical direction of the moving object intersect. And based on the reference position, the first part of the object placed on the first side with respect to the reference plane including the moving direction and the vertical direction, and the opposite to the first side with respect to the reference plane. To discriminate from the second part of the object arranged on the second side and to execute. The program may be provided recorded on a computer-readable storage medium (eg, non-transitory tangible media).

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. In addition, as long as it is permitted by law, the disclosure of Japanese Patent Application No. 2018-143687 and all the documents cited in the above-described embodiments will be incorporated as part of the description of the main text.

1 ... Detection device, 2 ... Position detection unit, 3 ... Processing device, 4 ... Detection unit, 5 ... Point cloud data generation unit, 11 ... Reference calculation unit, 12 ... -Partial discrimination unit, 13 ... posture estimation unit, 14 ... storage unit, 21 ... direction calculation unit, 22 ... model generation unit, BF ... reference plane, BP ... reference position, BP1 ... 1st reference position, BP1a ... 1st reference position, BP1b ... 1st reference position, BP2 ... 2nd reference position, CL ... center line, D2 ... point cloud data , M ... object (human body), M1 ... first part (left half body), M2 ... second part (right half body), MD ... movement direction, SL ... ground line, V1 ...・ ・ 1st threshold, V2 ・ ・ ・ 2nd threshold

Claims (25)

A detector that detects the position information of each point of a moving object,
A reference calculation unit that calculates a position where the amount of change in the position information satisfies a predetermined condition as a reference position in an intersection direction that intersects the moving direction and the vertical direction of the object.
With respect to the first portion of the object arranged on the first side with respect to the reference plane including the movement direction and the vertical direction based on the reference position calculated by the reference calculation unit, and with respect to the reference plane. A detection device including a discriminating unit for discriminating a second portion of the object arranged on the second side opposite to the first side. The reference position is the position of the portion including the center of the object in the crossing direction.
The detection device according to claim 1. The detection unit detects point cloud data including three-dimensional coordinates of each point on the surface of the object as the position information.
The reference position is determined based on the point cloud data.
The detection device according to claim 1 or 2. The reference position is based on the first reference position where the amount of change in the coordinates of the point cloud data is equal to or greater than a predetermined value.
The detection device according to claim 3. For the reference position, the maximum value of the coordinates in the vertical direction at each coordinate in the intersection direction is calculated for the points included in the point cloud data, and the amount of change of the maximum value with respect to the coordinates in the intersection direction is equal to or more than a predetermined value. Is the first reference position.
The detection device according to claim 4. The reference position is a second reference position within a predetermined region determined by the first reference position.
The discrimination unit discriminates a portion arranged on the first side with respect to the second reference position as the first portion, and determines a portion arranged on the second side with respect to the second reference position. Determine as the second part,
The detection device according to claim 4 or 5. The first reference position includes the position on the first side and the position on the second side of the object.
The area between the position on the first side and the position on the second side is defined as the predetermined area.
The second reference position is obtained from the predetermined region.
The detection device according to claim 6. The reference calculation unit calculates the minimum value of the coordinates of the points included in the point cloud data with the front of the moving direction as positive in each coordinate of the intersection direction, and the change of the minimum value with respect to the coordinates of the intersection direction. Select the second reference position based on the quantity,
The detection device according to claim 6 or 7. The reference calculation unit calculates the position where the minimum value is maximum as the second reference position.
The detection device according to claim 8. The discriminating unit discriminates between the first portion and the second portion by using the surface including the second reference position as the reference plane.
The detection device according to any one of claims 6 to 9. The discriminating unit discriminates between the first portion and the second portion based on the relative position between the geodesic line connecting the reference position and the portion of the object and the reference plane.
The detection device according to any one of claims 1 to 10. The discrimination unit includes a first feature amount of a portion arranged on the first side of the reference plane on the geodesic line and a portion arranged on the second side of the reference plane on the geodesic line. The relative position between the geodesic line and the reference plane is specified based on the second feature amount of the above, and the first portion and the second portion are specified based on the relative position between the specified geodesic line and the reference plane. To determine,
The detection device according to claim 11. The first feature amount is an average of the distances between each point and the reference plane for a plurality of points on the geodesy line arranged on the first side with respect to the reference plane.
The second feature amount is the average of the distances between each point and the reference plane at a plurality of points on the geodesy line arranged on the second side with respect to the reference plane.
The detection device according to claim 12. The first feature amount is the distance between the reference plane and the point on the geodetic line farthest from the reference plane to the first side.
The second feature amount is the distance between the reference plane and the point on the geodetic line farthest from the reference plane to the second side.
The detection device according to claim 12 or 13. A model generation unit that calculates model information of the object based on the first portion and the second portion determined by the discrimination unit is provided.
The detection device according to any one of claims 1 to 14. A posture estimation unit that estimates the posture of the object based on the first portion and the second portion determined by the discrimination unit is provided.
The detection device according to any one of claims 1 to 15. In the object, the first portion and the second portion alternately move in the moving direction.
The detection device according to any one of claims 1 to 16. The object includes the human body
The first part is at least a part of the left half of the body.
The second part is at least a part of the right half of the body.
The detection device according to any one of claims 1 to 17. As the reference position, the reference calculation unit calculates a first reference position corresponding to the head of the human body based on the amount of change in the position information from the arm to the head of the human body.
The detection device according to claim 18. The reference calculation unit uses the first reference position corresponding to the left half of the human body and the first reference position corresponding to the right half of the human body as the reference position of the human body in the crossing direction. Calculate the second reference position corresponding to the position of the center line,
The detection device according to claim 19. A direction calculation unit for calculating the moving direction of the object based on the detection result of the detection unit is provided.
The reference calculation unit calculates the reference position based on the movement direction calculated by the direction calculation unit.
The discriminating unit discriminates between the first portion and the second portion based on the moving direction calculated by the direction calculating unit.
The detection device according to any one of claims 1 to 20. The detection unit detects the position information of each point in the target area including the object, and extracts the position information of each point on the surface of the object from the position information of each point in the target area.
The detection device according to any one of claims 1 to 21. A reference calculation unit that calculates a position where the amount of change in position information of each point on the surface of the object satisfies a predetermined condition as a reference position in an intersection direction that intersects the moving direction and the vertical direction of the moving object.
Based on the reference position, the first portion of the object arranged on the first side with respect to the reference plane including the moving direction and the vertical direction, and the first portion of the object with respect to the reference plane. A processing device including a discriminating unit for discriminating a second portion arranged on a second side opposite to the first side. Detecting the position information of each point of a moving object and
Calculation of a position where the amount of change in the position information satisfies a predetermined condition in the intersection direction where the moving direction of the object and the vertical direction intersect is used as a reference position.
The first part of the object arranged on the first side with respect to the reference plane including the moving direction and the vertical direction based on the reference position, and opposite to the first side with respect to the reference plane. A detection method including discriminating from a second portion of the object arranged on the second side of the object. On the computer
In the crossing direction where the moving direction and the vertical direction of the moving object intersect, the position where the amount of change in the position information of each point on the surface of the object satisfies a predetermined condition is calculated as a reference position.
Based on the reference position, the first portion of the object arranged on the first side with respect to the reference plane including the moving direction and the vertical direction, and the opposite to the first side with respect to the reference plane. A processing program for discriminating from the second part of the object arranged on the second side and executing the operation.

Images