JP2011257244A - Body shape detection device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method that eliminates the need to set a threshold which does not depend upon the size of a shape and depends upon an object when detecting a shape which is large in number of parameters like a sphere, a cylinder, or a cone from a distance image and surface shape data.SOLUTION: A body shape detection device includes an input device, a detection processing unit, and a cylinder and cone detection unit. The input device acquires input data of distance image or surface shape dot group data of an object. The detection processing unit detects a plane shape, a sphere shape, and respective parameters from the input data. The cylinder and cone detection unit calculates shape parameters of one of a cylinder, a cone and a truncated cone using a detection result of detection means.

Description

  Embodiments described herein relate generally to a technique for detecting an object shape from a three-dimensional measurement result of an object.

  Conventionally, as a method for detecting an object shape by image processing, there is an object shape detection method using a Hough transform. In this method, a specific shape of an object is detected from a luminance image or a distance image obtained by a camera or a range finder. Specifically, this is a method of voting on parameter values of shapes that can be configured by each pixel, and setting parameter values having a large number of votes as parameter values of detected shapes.

  As a specific example, for example, when a planar shape is detected from a distance image, the planar shape is defined by a total of three parameters: two parameters for determining the normal direction of the plane and the distance between the plane and the origin. Each pixel votes for the coordinates in the three-dimensional voting space corresponding to the parameter values of all the planes passing through the coordinates, and detects the planar shape represented by the parameter values of the coordinates having the largest number of votes.

IEICE Technical Report, Vol. 104, No. 489 (MVE2004 40-49), pp. 27-32, Kunishi Yamamoto, Atsushi Nakazawa, Kiyoshi Kiyokawa, Haruo Takemura, Hough transform and expected value maximization Parameter estimation of quadratic surface model

  When a shape having a large number of parameters such as a cylinder or a cone is detected by a conventional object shape detection method, the number of dimensions of the parameter space for voting becomes high, so that it is difficult to implement the software on a computer. As a solution to this, there is a method of detecting a shape using a voting space having a small number of dimensions by dividing voting and detection processing into a plurality of stages and suppressing the number of parameters required at a time.

  In this method, in order to detect a cylinder and a cone by the Hough transform, first, the center of the sphere is detected, and the number of votes is compared with a predetermined threshold value to determine whether it is a sphere or a part of a cylinder and a cone. Determine. That is, it is necessary to set an appropriate determination threshold value and binarization threshold value depending on the resolution of the distance image and the size of the target object in the image. In addition, in the detection method by voting, when a plurality of shapes having different sizes appear in the input distance image, the larger the surface area, the larger the number of votes from pixels, so that larger objects tend to be detected more easily. is there.

  Therefore, when detecting shapes with a large number of parameters such as spheres, cylinders, and cones from distance images and surface shape data, a method that does not depend on the size of the shape and does not require target-dependent threshold setting. There is a problem of realizing.

  According to the present embodiment, the object shape detection apparatus includes an input unit, a detection unit, and a cylinder / cone detection unit. The input unit obtains input data of a distance image of the object or surface shape point cloud data. The detecting means detects a planar shape, a spherical shape, and each parameter from the input data. The cylinder / cone detection means calculates a shape parameter of any one of the cylinder, the cone, and the truncated cone using the detection result of the detection means.

The block diagram for demonstrating the structure of the object shape detection apparatus regarding embodiment. The flowchart for performing the rough description of the cylinder and cone detection process regarding embodiment. The flowchart for demonstrating the cylinder and cone detection process regarding embodiment. The flowchart for demonstrating the spherical body detection process regarding embodiment. The figure for demonstrating the voting processing method regarding embodiment. The flowchart for demonstrating the spherical body determination method regarding embodiment. The flowchart for demonstrating the plane detection process regarding embodiment. The flowchart for demonstrating operation | movement of the object identification apparatus regarding the application example of embodiment. The figure for demonstrating the structure of the object identification apparatus regarding the application example of embodiment. The figure which shows the detection result of the object identification apparatus regarding the application example of embodiment. The figure for demonstrating the structure of the fruit detection apparatus regarding the application example of embodiment.

  Embodiments will be described below with reference to the drawings.

[Configuration of Object Shape Detection Device]
FIG. 1 is a block diagram for explaining a configuration of an object shape detection apparatus according to the embodiment.

  As illustrated in FIG. 1, the object shape detection apparatus includes an image input device 10, an input interface 11, a display output device 12, a control unit (controller) 13, and an image processing unit 14.

  The image input device 10 includes, for example, a stereo camera or a range finder, and acquires a distance image of an object to be detected. The input interface 11 transfers the distance image input by the image input device 10 to the image processing unit 14. The controller 13 controls the entire apparatus, and in particular, inputs a processing result (recognition result) output from the image processing unit 14 and controls the display output apparatus 12 to display it on the screen.

  The image processing unit 14 includes an input / output interface 15, a plane detection processing unit 16, a sphere detection / determination processing unit 17, and a cylinder / cone detection processing unit 18. The input / output interface 15 provides the distance image transferred from the input interface 11 to each of the plane detection processing unit 16, the sphere detection / determination processing unit 17, and the cylinder / cone detection processing unit 18.

  The plane detection processing unit 16 performs processing for detecting, extracting, and estimating the position and orientation of the plane shape of the object based on the distance image. The sphere detection / determination processing unit 17 executes processing for detecting, extracting, and estimating the position and orientation of the sphere of the object based on the distance image. The cylinder / cone detection processing unit 18 executes processing for detecting, extracting, and estimating the position / orientation of the cylinder / cone shape of the object based on the distance image.

[Operation of object shape detection device]
The operation of the object shape detection apparatus according to the embodiment will be described below with reference to the flowcharts of FIGS.

  First, the outline of the cylinder (column) / cone detection process according to the present embodiment will be described with reference to the flowchart of FIG.

  The cylinder (column) and the cone (including the truncated cone) have 5 and 6 parameters representing the shape, respectively. For this reason, it is almost impossible to obtain directly by the Hough transform. Therefore, the image processing unit 14 according to the present embodiment uses the fact that a large number of planes and spheres are detected on a cylinder and a cone, and detects the plane and the distribution of the spheres.

  When vertices on a cylinder or cone are detected as part of a sphere, the vertices are distributed in a band around the axis of the cylinder / cone on each sphere. On the other hand, when these vertices are detected as a plane, the plane has an elongated strip shape along the axis. That is, since the belt-like vertex groups of the sphere and the plane are orthogonal to each other, the cylinder / cone is detected by combining them.

  In the image processing unit 14, the plane detection processing unit 16 and the sphere detection / determination processing unit 17 each detect a plane and a sphere from the input data (steps S1 to S3). Here, the input data is point cloud data representing the shape of the object surface as a point cloud, and is generated from the distance image. Further, the sphere detection / determination processing unit 17 acquires and classifies sphere information to be classified into a part of a cylinder / cone and a sphere based on the distribution of vertices on the sphere (step S4). The cylinder / cone detection processing unit 18 uses a sphere and a plane classified as a part of the cylinder / cone, and collectively detects those having vertices shared with each other (step S5). Further, the cylinder / cone detection processing unit 18 calculates the parameters of the cylinder / cone from the arrangement and radius of the sphere centers (step S6). If there are no vertices shared by each other, the cylinder / cone detection processing unit 18 calculates a plane parameter (step S7). In addition, the sphere detection / determination processing unit 17 calculates sphere parameters (step S8).

  Furthermore, with reference to the flowchart of FIG. 3, the cylinder (column) / cone detection processing according to the present embodiment will be described in detail.

  The image processing unit 14 inputs the distance image acquired by the image input device 10 through the input / output interface 15 as input data (step S21). The plane detection processing unit 16 detects a plane from the input data and calculates (estimates) the plane parameter (step S22). Thereby, the plane detection processing unit 16 extracts plane shape points constituting the plane from the input data (step S23).

  In parallel with this process, the sphere detection / determination processing unit 17 detects the sphere or a part thereof from the input data, and calculates (estimates) those parameters (step S24). Thereby, the sphere detection / determination processing unit 17 extracts sphere shape points constituting the sphere (step S25). Furthermore, the sphere detection / determination processing unit 17 determines whether each detected sphere is a cylinder or a part of a cone (step S26). When the sphere detection / determination processing unit 17 determines that the detected sphere is not a part of a cylinder or a cone, the sphere detection / determination processing unit 17 proceeds to a sphere detection process or a noise determination process described later.

  The image processing unit 14 proceeds to the next second stage process after the first stage process of detecting the plane and the sphere, calculating the respective parameters, and extracting the plane shape point and the sphere shape point. In other words, the cylinder / cone detection processing unit 18 detects a cylinder (column) and a cone (including a truncated cone) using the parameters and shape points acquired by the first stage process, and calculates (estimates) the parameters. (Steps S27 and S28).

  That is, the cylinder / cone detection processing unit 18 detects sharing (overlap) of shape points for the plane and the sphere, and sets a group of planes and spheres with overlap as one group. For example, when there are plane A, plane B, plane C, and sphere a, sphere b, and sphere c, plane A and sphere a, plane B and sphere a, plane C and sphere b, plane C and sphere c, respectively When the shape points are shared, plane A, plane B, and sphere a are set as one group, and plane C, sphere b, and sphere c are set as another group.

  Next, the cylinder / cone detection processing unit 18 calculates a shape parameter of any of a cylinder, a cone, and a truncated cone using the plane and sphere parameters in the group. Further, as post-processing, the cylinder / cone detection processing unit 18 extracts the shape points of the plane and the sphere in the group as shape points constituting the cylinder or the cone (step S29). Further, the cylinder / cone detection processing unit 18 corrects the shape parameter by a method such as a least square method and outputs the result as a detection result and a shape position / orientation estimation result (steps S30 and S31).

  Next, the sphere detection processing (steps S24 and S25) in the sphere detection / determination processing unit 17 will be specifically described with reference to the flowchart of FIG. 4 and FIG.

  Here, it is assumed that normal direction information of the shape surface is given in advance to each pixel of the distance image that is input data acquired by the image input device 10. When the sphere detection / determination processing unit 17 inputs the distance image as input data, the sphere detection / determination processing unit 17 executes a voting process in which each pixel of the distance image votes for a voting space formed by the sphere center coordinates in the three-dimensional space (Step S41, S42).

  FIGS. 5A and 5B are diagrams for explaining a voting processing method. As shown in FIGS. 5A and 5B, the shape point 401 on the surface shape represented by each pixel is voted for each coordinate position in the voting space representing the position 403 in the direction opposite to the normal direction 402 thereof. . After voting for all the shape points 401 is completed, the sphere center position 404 has more votes than others.

  The sphere detection / judgment processing unit 17 extracts the coordinates where the number of votes is, for example, the maximum value, the maximum value, or the threshold value and sets the coordinates as the detected sphere center position 404 (step S43). Next, the sphere detection / determination processing unit 17 estimates the radius of the sphere from the shape points around the sphere center position 404 or the distribution of the shape points voted for the center position 404 (step S44). As the radius estimation method, for example, a method of detecting a peak from a histogram of the distance between the sphere center and the shape point is used.

  Next, the sphere detection / determination processing unit 17 extracts the sphere shape points constituting the sphere whose parameters are estimated (step S45). Furthermore, the sphere detection / determination processing unit 17 corrects the sphere parameter using a method such as a least square method, and outputs the corrected sphere parameter and the sphere shape points constituting the sphere parameter (steps S46 and S47).

  Here, in the voting processing method of the sphere detection / determination processing unit 17, a relatively large sphere 406 and a small sphere 407 are included in the same distance image as input data, as shown in FIGS. There may be a plurality of spheres having different sizes. The large sphere 406 is composed of more shape points 401 than the small sphere 407. For this reason, when each shape point 401 is voted with a weight equal to the voting space, the number of votes is larger at the center position 404 of the large sphere 406.

  Therefore, by assigning voting weights according to the distance between the shape point of the voting source and the coordinates in the voting space of the voting destination, the same number of votes can be acquired regardless of the size of the sphere. Specifically, the sphere 406 having a larger radius (distance between the sphere center and the surface shape) is voted little by little from the many shape points 401 by voting with a smaller weight as the distance from the shape point 401 increases. On the other hand, the sphere 407 with a small radius obtains a large number of votes from a small shape point 401. In this way, it is not the area where the sphere is visible that affects the number of votes, but the proportion of the part where the sphere is visible.

  Next, with reference to the flowchart of FIG. 6, the sphere determination process (the process of step S26) in the sphere detection / determination processing unit 17 will be specifically described.

  The sphere detection / determination processing unit 17 inputs a sphere parameter obtained by the sphere detection process (step S24) and a sphere shape point group obtained by the sphere shape point extraction process (step S25) (step S51). The sphere detection / determination processing unit 17 calculates the normal direction of the approximate plane by the least square method using the input shape point group (step S52). When the sphere is a part of a cylinder or a cone, the distribution of the sphere-shaped point group is distributed on a circle centered on one point on the main axis, so that the normal direction is almost equal to the main axis.

  Next, the sphere detection / determination processing unit 17 compares the normal direction of each sphere shape point with the obtained normal direction of the approximate plane, and obtains the distribution of angles formed by the normal direction (step S53). That is, the sphere detection / determination processing unit 17 calculates the average and variance of the formed angles. If the sphere is a part of a cylinder or a cone, the normal direction of the sphere shape point is distributed radially with an almost equal angular difference from the main axis.

  Accordingly, when the variance of the formed angles is less than the threshold value, the sphere detection / determination processing unit 17 determines that it is a part of a cylinder or a cone (NO in step S54). On the other hand, the sphere detection / determination processing unit 17 determines that the sphere is a single sphere when the variance of the formed angles is equal to or greater than the threshold (YES in step S54, S56).

  On the other hand, when the sphere detection / determination processing unit 17 determines that the cylinder is a part of a cylinder or a cone, the sphere detection / determination processing unit 17 determines that the cylinder is a cylinder if the average angle formed is nearly orthogonal (YES in step S55, S57). ). In the case of a cylinder, all normal directions are almost perpendicular to the main axis, whereas in the case of a cone, it can be biased. Therefore, the sphere detection / determination processing unit 17 determines that the cone is a cone when the average angle formed is not orthogonal (NO in step S55, S58). However, the cylindrical or conical determination process can be omitted because it is not always necessary in the subsequent processes.

  Next, with reference to the flowchart of FIG. 7, the plane detection process (step S22) and the plane shape point extraction process (step S23) in the plane detection processing unit 16 will be specifically described.

  Here, it is assumed that normal direction information of the shape surface is given in advance to each pixel of the distance image that is input data acquired by the image input device 10. When the plane detection processing unit 16 inputs a distance image as input data, each pixel of the distance image votes based on the normal direction to vote in a voting space having parameters representing the normal direction of the plane as axis components. Processing is executed (steps S61 and S62). The voting space may be a two-dimensional space that represents only the normal direction of the plane or a three-dimensional space that represents the distance between the normal direction of the plane and the origin. Voting to the two-dimensional space of only the normal direction is equivalent to a process of quantizing the normal direction and grouping shape points having the same normal direction.

  After the voting of all the shape points is completed, the plane detection processing unit 16 extracts coordinates that satisfy the maximum or the threshold value or both from the voting space, and detects a plane using the coordinates as parameters (step S63). ). Further, the plane detection processing unit 16 extracts a shape point whose distance is within a certain distance from the detected plane or a shape point voted for its plane parameter (step S64). Next, the plane detection processing unit 16 divides each connected area as another plane based on the adjacent information between the shape points (step S65). In this way, the plane detection processing unit 16 is configured with shape points having normal directions within a certain range, each connected region is detected as one plane, and the plane parameters and the extracted shape points are detected. Is output (step S66).

  As described above, according to the present embodiment, a shape having a large number of parameters such as a cylinder and a cone can be detected from a distance image and surface shape data without setting an object-dependent threshold. Even when a plurality of spheres having different sizes exist in the distance image or the surface shape data, the sphere can be detected regardless of the size. In addition, since spheres of different sizes are detected with the same degree of importance, it is possible to stably detect cones having different circular radii depending on the height by using this. Furthermore, by reducing the number of dimensions of the parameter space for voting, it is possible to facilitate implementation by software on a computer.

[Application example]
8 to 10 are diagrams for explaining an object identification device to which the object shape device of the embodiment is applied.

  As shown in FIG. 9, the object identification device is connected to a shape measuring device 815 such as a line sensor installed on the road side, for example. The shape measuring device 815 measures the shape of the vehicles 801 and 802 passing through the road. The vehicle 801 has six tires 803 to 808 attached to the main body. The vehicle 802 also has six tires 809 to 814 attached to the main body. Each of the vehicles 801 and 802 is equipped with a fuel tank as shown in FIG.

  As shown in the flowchart of FIG. 8, the object identification device inputs shape data (distance image) measured by the shape measurement device 815 as input data (step S71). The object identification device uses the input data to execute a cylinder (column) / cone detection process including steps S22 to S30 shown in FIG. 3 of the embodiment (steps S72 to S80).

  That is, the object identification device outputs a plane detection result indicating the plane shapes of the tires and the fuel tanks of the vehicles 801 and 802 by plane detection / extraction processing (steps S72 and S73). Further, the object identification device outputs a sphere detection result indicating the sphere shape of the tires and fuel tanks of the vehicles 801 and 802 by the sphere detection / extraction process (steps S74 and S75). Further, the object identification device outputs a cylinder / cone detection result indicating the cylinder and cone shape of the tires and fuel tanks of the vehicles 801 and 802 by the cylinder / cone detection / extraction process (steps S77 to S80).

  Using these detection results, as shown in FIGS. 10 (A) to 10 (C), the object identification device can obtain a partial object shape that can be approximated by a planar shape or a cylindrical shape such as tires 901 and 903 and a fuel tank 902. Is detected, and a vehicle type determination process for determining the vehicle type based on the arrangement of these partial object shapes is executed (step S81). The object identification device outputs a vehicle type discrimination result by the vehicle type discrimination process (step S82).

  As described above, it is possible to discriminate the vehicle types of the vehicles 801 and 802 passing through the road by the object identification device to which the object shape device of the embodiment is applied. In addition to the planar shape and the cylindrical shape, the object identification device executes a vehicle type determination process using a detection result of a vehicle type that includes a partial object shape that can be approximated by a sphere, a cone, or a truncated cone.

  FIG. 11 is a diagram for explaining a plant monitoring device to which the object shape device of the embodiment is applied.

  As illustrated in FIG. 11, the plant monitoring device is mounted on, for example, a vehicle 1004 and includes a distance image input device 1001, an image processing device 1002, and an information recording device 1003. According to the movement of the vehicle 1004, the plant monitoring device detects the fruit 1006 that grows in the fruit tree 1005 while moving in the orchard.

  Specifically, the plant monitoring apparatus captures a range image including a fruit 1006 using the range image input device 1001. The image processing apparatus 1002 detects and extracts a sphere shape and calculates a position / size and the like by a sphere detection / extraction process (steps S24 and S25) as shown in FIG. 3 of the embodiment. The information recording device 1003 stores the processing result of the image processing device 1002.

  The plant monitoring device to which the object shape device of the embodiment as described above is applied can detect the fruit of the orchard and accumulate the detection result in the information recording device 1003. Therefore, the information stored in the information recording apparatus 1003 can be used to count the fruit in the orchard and estimate the position and size of the fruit in the orchard. It is also possible to use the information accumulated over several years to recognize the growth and distribution of fruits in an orchard and to predict the production of the crop in that year at an early stage.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 10 ... Image input device, 11 ... Input interface, 12 ... Display output device,
13 ... Control unit (controller), 14 ... Image processing unit, 15 ... Input / output interface,
16... Plane detection processing unit, 17.
18 ... Cylinder / cone detection processing unit, 801,802 ... vehicle, 815 ... shape measuring device,
1001 ... Distance image input device, 1002 ... Image processing device, 1003 ... Information recording device,
1004 ... Vehicle.

Claims (13)

An input means for acquiring input data of an object distance image or surface shape point cloud data;
Detecting means for detecting a planar shape, a spherical shape and each parameter from the input data;
An object shape detection apparatus comprising: a cylinder / cone detection means for calculating a shape parameter of any one of a cylinder, a cone, and a truncated cone using a detection result of the detection means. The detection means includes
Plane detection means for detecting a plane from the input data, calculating plane parameters, and extracting plane shape points constituting the plane;
2. A sphere detection unit that detects a sphere or a part thereof from the input data, calculates parameters thereof, and extracts a sphere shape point constituting the sphere. Object shape detection device. The cylinder / cone detection means includes:
Means for detecting the presence or absence of a shape region shared by a plane and a part of a sphere;
Means for extracting a plane and a sphere having a shared shape area as one group;
Means for calculating a shape parameter of any one of a cylinder, a cone, and a truncated cone using a parameter of a plane and a sphere in the group, or a pixel or a shape point constituting the plane and a sphere in the group, or both; The object shape detection apparatus according to claim 1, wherein the object shape detection apparatus is configured to include: The cylinder / cone detection means includes:
As post-processing after calculating the shape parameter,
Means for extracting shape points of the plane and sphere in the group as shape points constituting a cylinder or a cone;
4. The structure according to claim 1, further comprising means for correcting the shape parameter by a method such as a least square method and outputting the result as a detection result and a shape position direction estimation result. The object shape detection apparatus described in 1. The sphere detecting means includes
The structure according to claim 2, further comprising means for determining whether a part of the sphere detected from the input data is a part of a cylinder, a cone, or a truncated cone. Object shape detection device. The sphere detecting means includes
Distribution of three-dimensional coordinates representing whether a part of a sphere detected from the input data is a part of a cylinder, a cone, or a truncated cone, and pixels or shape points included in the part of the sphere The object shape detection apparatus according to claim 2, wherein the object shape detection apparatus includes a determination unit based on a distribution in the normal direction. An input means for acquiring input data of an object distance image or surface shape point cloud data;
Voting processing means for voting to a voting space in which each pixel or each shape point in the input data includes a parameter representing the center position of a sphere as each axis component;
Sphere detection means for outputting a sphere represented by a parameter value selected based on the distribution of the number of votes by the voting processing means as a detection result;
The sphere detecting means includes
Object shape detection characterized by weighting a value to be voted in accordance with a distance between a three-dimensional coordinate value represented by a voting source pixel or a shape point and a three-dimensional coordinate value represented by a coordinate in a voting destination voting space apparatus. The sphere detecting means includes
In the weighting process, the weighting process is performed so that the smaller the distance between the three-dimensional coordinate value represented by the voting source pixel or the shape point and the three-dimensional coordinate represented by the coordinate value of the voting destination voting space, the smaller the value is voted. The object shape detection apparatus according to claim 7. The sphere detecting means includes
When the minimum value of the sphere radius is set and the distance between the three-dimensional coordinate value represented by the voting source pixel or the shape point and the three-dimensional coordinate value represented by the coordinates of the voting destination voting space is less than the minimum value, 9. The object shape detection apparatus according to claim 7, wherein the voting process is not executed or a value smaller than a value determined by the weighting process is voted. An input means for acquiring input data of an object distance image or surface shape point cloud data;
Voting processing means for voting to a voting space in which each pixel or each shape point in the input data includes a parameter representing a normal direction of a plane as each axis component;
Plane detection means for outputting a plane represented by a parameter value selected based on the distribution of the number of votes by the voting processing means as a detection result;
The plane detection means includes
An object shape detection apparatus that extracts connected regions as one plane based on adjacent information of pixels or shape points included in the detection result plane. Means for detecting a partial object shape of a vehicle using the object shape detection device according to any one of claims 1 to 6;
An object identification device comprising vehicle type discrimination means for identifying the type of vehicle based on the detection result. Means for detecting the shape of a plant using the object shape detection device according to any one of claims 7 to 9;
An information storage means for storing information for identifying a plant based on the detection result. Processing to obtain input data of an object distance image or surface shape point cloud data;
Processing for detecting a planar shape, a spherical shape, and each parameter from the input data;
And a process of calculating a shape parameter of any one of a cylinder, a cone, and a truncated cone using the detection result of the detection means.

Images