CN105513040A - Shapeless body position presumption method, and shapeless body position presumption device and program - Google Patents
Shapeless body position presumption method, and shapeless body position presumption device and program Download PDFInfo
- Publication number
- CN105513040A CN105513040A CN201510518788.XA CN201510518788A CN105513040A CN 105513040 A CN105513040 A CN 105513040A CN 201510518788 A CN201510518788 A CN 201510518788A CN 105513040 A CN105513040 A CN 105513040A
- Authority
- CN
- China
- Prior art keywords
- region
- threshold value
- overlap
- presumption
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention provides a shapeless body position presumption method, and a shapeless body position presumption device and program, and the overlapped upper and lower relationship of a shapeless body the whole shape of which can deform easily can be presumed in high precision. A distance sensor (13) mainly uses a measuring part (3) to measure the distance information, even if the shapeless body has a transparent part, the position of the shapeless body can be presumed in high precision. Furthermore, a threshold setting part (23) sets a threshold used for distinguishing overlapped parts, and an overlap mode presumption part presumes the overlap mode and presumes which overlap mode exists to obtain an object candidate. An upper and lower presumption part (27) presumes which object is at the uppermost side by means of a plurality of evaluation indexes. A centre-of-gravity position presumption part (29) presumes the backup centre-of-gravity position of the object arranged on the upper part.
Description
Technical field
The present invention relates to unsetting body position presuming method, unsetting body position estimating device and program, particularly relate to the unsetting body position presuming method etc. estimated up and down of the overlap to multiple indefinite body.
Background technology
In recent years, the exploitation of bin pickup (binpicking) system employing mechanical arm has been carried out.Bin pickup refers to and hold object successively among arbitrarily stacked multiple object, and is carried to specific place.In bin pickup, near the centre of gravity place of wishing holding object, therefore the centre of gravity place information of object is important.
Under, the state having multiple object in bulk, centre of gravity place presumption is carried out at memory box.These objects because of carrying in vibration, impact etc. and in casing deflection, overlap.When so producing deflection, overlap, need to carry out corresponding process respectively.Particularly when object overlap each other and produce cover, be difficult to the detection carrying out picking up object.
In the past, the feature etc. of geometric configuration is utilized to estimate the centre of gravity place of the detected object thing as rigid body.In addition, in non-patent literature 1, record following content: for unsetting workpiece, utilize 3-D view identification to obtain positional information.
Non-patent literature 1:AplliedVisionSystemsCorporation, " real time 3-D image process unsetting workpiece picking up system in bulk ", [online], internet <URL=" http://avsc.jp/images/pdf/Picking-system.pdf " >
But, such as when the indefinite body that global shape easily changes using the bag-shaped workpiece of softness of the local transparent as storage food, small articles, surface is atypic curved surface is as object, due to different from rigid body, its shape can change, so cannot utilize geometric configuration etc.Therefore, be that the method for object cannot be applied to indefinite body with rigid body.
In addition, the method described in non-patent literature 1 utilizes 3-D view identifying processing.Therefore, such as using the unsetting workpiece with large plane such for rice bag as object, by carry out plane detection thus obtain hold position.The global shape of indefinite body easily produces distortion, and surface is generally atypic curved surface.Therefore, the method described in non-patent literature 1 is the method for unsetting workpiece as object local with plane, is difficult to be applied to the yielding common indefinite body of overall appearance.
Summary of the invention
Thus, the unsetting body position presuming method etc. that the object of the present invention is to provide a kind of upper and lower relation that can hold the overlap of yielding indefinite body to global shape accurately to estimate.
The first aspect of the present application is the unsetting body position presuming method estimated up and down of the overlap to multiple indefinite body, and comprising: determination step, distance determination unit is to the area test distance that there is described multiple indefinite body; Threshold value setting procedure, the threshold value that the region that threshold value assigning unit assigns is used for creating the overlap of described multiple indefinite body in the region measured described distance determination unit is distinguished with the region do not overlapped; And estimate step up and down, estimate that unit uses described distance determination unit to determine up and down with the distance of described multiple indefinite body, the evaluation index that the part high according to the described threshold value of contrast and/or the part lower than described threshold value carry out evaluating carries out the presumption up and down of the overlap of described multiple indefinite body.
The second aspect of the present application is the unsetting body position presuming method of first aspect, comprise overlap scheme presumption step, the region that overlap scheme presumption unit measures described distance determination unit, object candidate is obtained according to the profile in object region, estimate up and down in step described, if there is multiple object candidate, then the described unit that estimates up and down is to described each object candidate, be used in when being positioned at top and become the evaluation index of high praise, be selected to the described object candidate of most high praise as the object being positioned at overlapping top.
The third aspect of the present application is the unsetting body position presuming method of second aspect, described estimate the region that threshold value is low described in ratio that evaluation index that unit uses comprises described object candidate up and down area, the volume of described object candidate, described object candidate region in gradient, the limit goodness of fit of described object candidate and the angle goodness of fit of described object candidate at least one.
The fourth aspect of the present application is a kind of unsetting body position estimating device estimated up and down of the overlap to multiple indefinite body, possesses: distance determination unit, and it is to the area test distance that there is described multiple indefinite body; Threshold setting unit, it sets the threshold value that the region creating the overlap of described multiple indefinite body in the region for measuring described distance determination unit is distinguished with the region do not overlapped; And estimate unit up and down, its use described distance determination unit to determine with the distance of described multiple indefinite body, the evaluation index that the part high according to the described threshold value of contrast and/or the part lower than described threshold value carry out evaluating carries out the presumption up and down of the overlap of described multiple indefinite body.
5th aspect of the present application is a kind of program, for make computing machine as threshold setting unit with estimate unit up and down and play function, described threshold value assigning unit assigns is used for existing in the region of multiple indefinite body to what determined distance by distance determination unit, the threshold value distinguished is carried out in the region creating the overlap of described multiple indefinite body and the region do not overlapped, described that estimate that unit uses described distance determination unit to determine up and down with the distance of described multiple indefinite body, the evaluation index that the part high according to the described threshold value of contrast and/or the part lower than described threshold value carry out evaluating carries out the presumption up and down of the overlap of described multiple indefinite body.
In addition, also the present application can be interpreted as the recording medium that Gong the computing machine of stable recording the 5th aspect reads.
In addition, distance determination unit can be the unit of the distance measuring this distance determination unit and multiple indefinite body, and also can be configured to, indefinite body is present on ground, and distance determination unit measures the distance of indefinite body apart from ground.
According to each side of the present application, by mainly utilizing range information, even transparent part also can estimate accurately, and, the place creating lap and the place not overlapping part is distinguished by setting threshold value, and utilize the relation in high place and low place to estimate upper and lower relation, thus evaluation can be carried out to the part of overlap and nonoverlapping part and estimate upper and lower relation accurately.
And then, according to the second aspect of the present application, by using template matches (templatematching) to obtain object candidate, using it also to use multiple evaluation index to estimate upper and lower relation, thus position detection can be carried out to upside object accurately.And then, according to the third aspect of the present application, by multiple evaluation number being combined particularly, high-precision position verification and measurement ratio can be realized.According to the experiment of inventor, when template matches success, the position verification and measurement ratio of upside object is 100%.
Accompanying drawing explanation
Fig. 1 is the block diagram of an example of the structure of the bin picking up system involved by embodiment representing the present application.
Fig. 2 is the process flow diagram of an example of the action of the signal conditioning package 5 representing Fig. 1.
Fig. 3 is the threshold value h of the step ST7 for illustration of Fig. 2
othe figure of concrete example of setting process.
Fig. 4 is the process flow diagram of an example of the presumption process of the overlap scheme of the step ST8 representing Fig. 2.
Fig. 5 is the figure of the concrete example of the presumption process of overlap scheme for illustration of Fig. 4.
Fig. 6 is key diagram 6 (a) rigid body and the different figure of the area under the projection of Fig. 6 (b) flexible body.
Fig. 7 represents the concrete example in flexible body, and Fig. 7 (a) illustrates volume, and Fig. 7 (b) illustrates gradient, and Fig. 7 (c) illustrates the limit goodness of fit.
Fig. 8, for actual Fig. 8 (a) color image of obtaining and Fig. 8 (b) range image, illustrates Fig. 8 (c) area, Fig. 8 (d) volume, Fig. 8 (e) gradient, Fig. 8 (f) limit goodness of fit, Fig. 8 (g) angle goodness of fit.
Fig. 9 is the process flow diagram of an example of the areal calculation process of the low area represented as evaluation index 1.
Figure 10 is the process flow diagram of an example of the volume computing process represented as evaluation index 2.
Figure 11 is the process flow diagram of an example of the Grad computing represented as evaluation index 3.
Figure 12 is the process flow diagram of an example of the limit goodness of fit computing represented as evaluation index 4.
Figure 13 is the figure representing the angle of extracting out from object.
Figure 14 is the process flow diagram of the example representing angle pump-and-treat system.
Figure 15 is the process flow diagram of an example of the angle goodness of fit computing represented as evaluation index 5.
Figure 16 represents the overlapping figure estimating the process process of experiment up and down.
Figure 17 represents estimating in 30 data prepared in experiment, output figure from 27 data of target jaw position to the immediate vicinity of upside object up and down.
The explanation of Reference numeral
1... bin picking up system; 3... determination part; 4... handle part; 5... signal conditioning package; 7... casing; 9... indefinite body; 11... color sensor portion; 13... range sensor portion; 15... arm control part; 17... arm; 21... control part; 23... threshold value configuration part; 25... overlap scheme presumption unit; 27... upper and lower presumption unit; 29... centre of gravity place presumption unit.
Embodiment
Below, with reference to accompanying drawing, carry out describing to the embodiment of the present application.In addition, the embodiment of the present application is not limited to following embodiment.
Embodiment
Fig. 1 is the block diagram of an example of the structure of the bin picking up system involved by embodiment representing the present application.Bin picking up system 1 possesses determination part 3, handle part 4 and signal conditioning package 5.In bin picking up system 1, measure in determination part 3 pairs of casings 7, when signal conditioning package 5 is judged as having indefinite body 9, handle part 4 carries out the process of taking out indefinite body 9.Repeatedly carry out processing till indefinite body 9 becomes and do not exist.In FIG, show and have two indefinite bodies 9 in casing 7
1and 9
2situation.Below, sometimes also subscript is omitted.Multiple indefinite body 9 is bulked in casing 7, deflection, overlap because of the vibration, impact etc. in carrying.In addition, also exist and fall in pickup midway and cause overlapping situation.
Indefinite body 9 is that global shape easily changes, and surface is the object of atypic curved surface.It is such as the bag-shaped workpiece of softness of storage food, small articles.But, in the present embodiment, suppose that the indefinite body 9 in identical casing is object of the same race, and the error of the size of indefinite body 9 and approximate dimensions given in advance is below 10%.Among soft bag-shaped workpiece, owing to also there is the workpiece of local transparent, so be unsuitable for the process utilizing simple image procossing to carry out.In addition, because global shape easily changes, so be difficult to use the method being assumed to definite shape such as described in non-patent literature 1.Therefore, the present application mainly uses the range information obtained by range sensor to carry out the presumption up and down of indefinite body 9, the centre of gravity place of the indefinite body of presumption top.
Take in determination part 3 pairs of casings 7.Determination part 3 possesses color sensor portion 11 and range sensor portion 13.Color sensor portion 11 is taken the image in casing 7 as common camera.Distance in 13 pairs, range sensor portion casing measures.As the sensor measuring distance, such as, there is camera distance, laser range finder (LRF), RGB-D sensor.In the experiment illustrated below, be used as the Kinect of RGB-D sensor.This Kinect is cheap sensor, and precision is lower compared with other sensors.As will be described later, even if utilize the low sensor of this precision also to present high detection rate, other range sensor is utilized can to obtain high-precision result too.
Handle part 4 holds the indefinite body 9 in casing 7.Handle part 4 possesses arm control part 15 and arm 17.Arm control part 15 controls the action of handle part 4.Arm 17 holds indefinite body 9 by the control of arm control part 15.Arm control part 15 obtains the centre of gravity place of the indefinite body 9 being positioned at top by signal conditioning package 5 and arm 17 is held.
Signal conditioning package 5 uses the measurement result of determination part 3 to estimate the indefinite body 9 being positioned at top, estimates the centre of gravity place of this indefinite body 9.Signal conditioning package 5 possesses control part 21, threshold value configuration part 23, overlap scheme presumption unit 25, up and down presumption unit 27 and centre of gravity place presumption unit 29.The action of control part 21 control information treating apparatus 5.Threshold value configuration part 23 sets the threshold value being used for distinguishing with the region do not overlapped the region of the overlap creating multiple indefinite body.Overlap scheme presumption unit 25 is carried out template matches for the region that determination part 3 measures thus is obtained object candidate.The object being positioned at top in upper and lower presumption unit 27 pairs of object candidates estimates.The centre of gravity place of centre of gravity place presumption unit 29 to the object candidate being estimated as top estimates.
Fig. 2 is the process flow diagram of an example of the action of the signal conditioning package 5 representing Fig. 1.First, to two threshold value h that Fig. 2 uses
lwith h
obe described.Threshold value h
lit is the threshold value distinguishing the region that there is indefinite body 9 and the region that there is not indefinite body 9.The region that there is not indefinite body 9 is the bottom surface of the casing 7 of the reference field become when obtaining elevation information.Consider the noise (noise) of sensor, with indefinite body 9 independently by threshold value h
lbe set to constant value.Below, h
l=15mm.Threshold value h
oit is the threshold value distinguishing region and other the region uprised because multiple indefinite body 9 is overlapping.Later setting is specifically described.
With reference to Fig. 2, an example of the action of the signal conditioning package 5 of Fig. 1 is described.Control part 21 sets threshold value h
l(step ST1).Then, control part 21 obtains determination data (sensor information) (the step ST2) that the color sensor portion 11 of determination part 3 and range sensor portion 13 determine.Obtaining from range sensor portion 13 take sensor as the range information of benchmark.Elevation information D (x) (y) that it is benchmark that control part 21 is converted into the height on the ground apart from casing.Here, bottom surface is set to xy plane, is set to z-axis by above casing.Control part 21 judges (step ST3) whether there is indefinite body 9.Such as, when existence is than threshold value h
lduring high value, be judged to be to there is indefinite body 9.If indefinite body 9 does not exist because all to take out etc. and become, then end process.
When having indefinite body 9, to whether having overlap to judge (step ST4).Such as determine whether overlap in the following manner.First, the maximal value d of elevation information is detected
mAX.Then, will from bottom surface to height d
mAXbetween be equally spaced divided into 5 parts, and the counting of certificate of counting of falling into a trap in each height region.The value of the lower limit in regions maximum for this number of data points is set to the altitude datum d of object.Region represented by the point group more than height of 1.5 times of this altitude datum d is set to and creates overlapping region.Next, such as in the following manner overlapping region is detected.Altitude datum d is set to threshold value, contrasts its high region and carry out cluster process, 1.5d is set to threshold value, the region higher than it is set to overlapping region and carries out cluster process, carry out associating of the position of the cluster higher than d and the cluster higher than 1.5d.Check long axis direction, the short-axis direction of the cluster higher than 1.5d.Overlapping object is created, by the short-axis direction being present in the cluster higher than 1.5d and the high cluster of the ratio d contacted with this cluster is set to overlapping region assuming that exist on the short-axis direction of the cluster higher than 1.5d.
When not having overlapping, centre of gravity place presumption unit 29 uses the methods such as such as template matches to estimate position and the posture of object, presumption centre of gravity place (step ST5).Template matches refers to: making search domain move to the position of the template consistent the most (coupling) of image information in this search domain and the preprepared object for retrieve, estimating position and the posture of the object that wish is retrieved.Then, step ST2 is got back to.
Depositing in a case of overlap, detecting (step ST6) creating overlapping region (overlapping region).Below, overlapping region is set to x
ox≤ x≤x
of, y
ox≤ y≤y
of.Then, threshold value configuration part 23 sets threshold value h
o(step ST7).Overlap scheme presumption unit 25 estimates overlap scheme (step ST8).Upper and lower presumption unit 27, according to the comprehensive descision of multiple evaluation index, carries out overlapping presumption up and down (step ST9).Then, centre of gravity place presumption unit 29 estimates the centre of gravity place x of the object of upside
g, y
g(step ST10).Then, step ST2 is got back to.
Then, with reference to Fig. 3, to the threshold value h of the step ST7 of Fig. 2
oan example of setting be described.S (h) is set to the quantity of the pixel of below elevation information h.By r
o(0 < r
o< 1) be set to the ratio representing that the template of the object deduced by mating is overlapping.Now, formula (1) is utilized to define threshold value h
o.Here, r
os (h
l) represent the area of overlapping region.By elevation information at h
oabove region is defined as because of object overlap each other and the region uprised.Here, r is set as follows
o.The size of template is set to S
t.Use Gauss's symbol [] and by r
obe set to formula (2).Be described for the overlap of Fig. 3.Fig. 3 (a) is overlapping example.The vertical view of Fig. 3 (a) is the figure from top view, and the side view of Fig. 3 (a) is the figure observed from the side.Fig. 3 (b) represents the area of the region entirety that there is overlapping object, is denominator and the S (h of formula (2)
l).Oblique line portion, bottom right and the lower-left oblique line portion of Fig. 3 (c) do not represent the area in region when not having overlapping, and the Section 1 of the molecule on the right of formula (2) is combined and the item obtained by the area of bottom right oblique line and lower-left oblique line.The colored portion of Fig. 3 (d) represents the area in the region uprised because of overlapping, the molecular entities of expression (2).The r of formula (2)
oit is overlapping ratio.
h
o＝{min(h)|S(h)≥r
oS(h
l)}(1)
Then, with reference to Fig. 4 and Fig. 5, an example of the presumption of the overlap scheme of the step ST8 of Fig. 2 is described.Fig. 4 is the process flow diagram of an example of the presumption process of intermediate scheme.Use threshold value h
lbinary conversion treatment is carried out to elevation information D (x) (y), thus overlapping region is distinguished as the region that there is object and the region that there is not object.By grasping the approximate location creating overlapping region, the region that there is object around it becomes the region of the overlap creating object.Therefore, the region that there is object is obtained.Here, the region that there is object refers to the region making elevation information become large because of object.Use the threshold value h that the elevation information for distinguishing this region is large or little
l.The region that there is object is the region that overlap has multiple object.Next, there is the area applications template matches of object for overlap, estimate object and be positioned at which position on two dimensional image.Process by mating the result obtained as object candidate.
The concrete example of Fig. 5 is used to be described.Fig. 5 (a) represents range image.Here, the square of solid color represents pixel.The color of pixel represents the region be split to form at equal intervals by elevation information.Region is represented by these 5 grades red, yellow, green, light blue, blue, the region that red expression is the highest, the region that blue expression is minimum.Use threshold value h
lobtain the region that there is object.In Fig. 5 (a), red, yellow, green is that height is at h
labove pixel, light blue, blueness is highly less than h
lpixel.Therefore, as shown in Fig. 5 (b), red, yellow, green are divided into the black as the region that there is object, light blue, blueness are divided into the white as the region that there is not object.Next, template matches is carried out to the region of the black that there is object region, thus as shown in Fig. 5 (c), obtain the region of the rectangle of red dotted line and these two object candidates of rectangle of light blue dotted line.
Then, with reference to Fig. 6 ~ Figure 15, the presumption up and down of the overlap that the comprehensive descision of multiple evaluation indexes of the step ST9 based on Fig. 2 carries out is described.When object is rigid body, due to indeformable, so can judge by a kind of method, without the need to deliberately combining.In the present embodiment, first, which type of overlap mode presumption overlap scheme, there is in presumption.Then, multiple evaluation index is used to be the object being positioned at top side so which to estimate.As the evaluation index estimated up and down, five indexs are described.They are gradient, the limit goodness of fit of object candidate and the angle goodness of fit of object candidate in the area of the low area of object candidate, the volume of object candidate, object candidate areas.Fig. 6 (a) and Fig. 6 (b) are the different figure be described for the area under the projection to rigid body and flexible body.Fig. 7 represents the concrete example in flexible body, and Fig. 7 (a) represents volume, and Fig. 7 (b) represents gradient, and Fig. 7 (c) represents the limit goodness of fit.Fig. 8, for actual Fig. 8 (a) color image of obtaining and Fig. 8 (b) range image, illustrates Fig. 8 (c) area, Fig. 8 (d) volume, Fig. 8 (e) gradient, Fig. 8 (d) limit goodness of fit, Fig. 8 (e) angle goodness of fit.
First, as evaluation index 1, the area of low area is described.Because of object overlap each other, the object of upside tilts, its projected area diminishes.Therefore, directly do not use elevation information, but define projected area based on elevation information, using this area as one of evaluation index.In Fig. 6 (a), thick line represents than threshold value h
olow region.Above-mentioned zone is to have two across the mode in the region higher than threshold value.Therefore, from left side by from directly over the length in this region of observing be set to L
lwith L
r.If by from directly over the length in object region of observing be set to L
t, then from directly over the length L in region of overlap that observes
obecome formula (3).Thus, if by from directly over the length in the region higher than height h of observing be set to L (h), then can obtain threshold value h according to formula (4)
o.With threshold value h
o, overlapping angle Φ (0 < Φ < pi/2) have nothing to do, L
r> L
lall the time set up.That is, the area that the area ratio of low area being positioned at the object of upside is positioned at the low area of the object of downside is little.
h
o＝[h|L(h)＝L
o](4)
According to Fig. 6 (b), even flexible body, this relation is also set up.L
tequal with rigid body at flexible body, therefore with L
tfor the L of parameter
obecome L
o*=L
o.Therefore, L
r* > L
r, L
l* < L
l, and L
r* > L
l*.According to above content, the area of low area is less, be then that the possibility of the object of upside is higher, area evaluation of estimate is set to larger.Fig. 9 is the process flow diagram of an example of the areal calculation process representing low area.
As an example, the overlap of Fig. 8 (a) is considered.Its range image has been shown in Fig. 8 (b).Fig. 8 (c) represents high region by lavender, represents the image of low area by blueness.Red dotted line and green dotted line are object candidates, and red object candidate represents the object being positioned at upside.In Fig. 8 (c), compared by the area of low area, red object candidate is less than green object candidate.
Then, as evaluation index 2, volume is described.Because of object overlap each other, the object of upside is caused to tilt.Thus, between the object and bottom surface of upside and between the object of upside and the object of downside, gap is produced.Thus, upside object apparent on volume become large.Therefore, not merely pay close attention to certain any elevation information, but use the elevation information of the region entirety of object candidate.Elevation information in region is added together obtained V and obtains as formula (5), represents the volume in the region of object candidate.There is the object region that then elevation information is larger being positioned at upside.Thus, compare the integration of the elevation information in candidate region and volume, the region list being positioned at the object of top side reveals high value.Such as, as shown in Fig. 7 (a), two object candidates that the presumption by overlap scheme obtains are compared.Here, L
tit is the width (therefore, equal with flexible body at the rigid body of supposition) of template.From Fig. 7 (a), the volume of candidate one side of the red dotted line uprised because of overlapping is large.Therefore, volume is larger, then this object region is that the possibility of the object being positioned at overlapping upside is higher.According to above content, be evaluation index with volume, volume is larger, be then that the possibility of the object of upside is higher, volume assessment value is set to larger.Figure 10 is the process flow diagram of the example representing volume computing process.
As an example, the overlap of Fig. 8 (a) is considered.Fig. 8 (d) represents high region by lavender, represents the image of low area by blueness.Red dotted line and green dotted line are object candidates, and red candidate represents the object being positioned at upside.Red candidate is compared with the candidate of green, and the ratio in high region is many, and therefore, the volume of red candidate is also large.
Then, as evaluation index 3, gradient is described.Because object exists thickness, so when creating overlap, the ladder surrounded by lavender producing such as Fig. 7 (b) on the border of object is poor.With this ladder difference for boundary, elevation information significantly changes.Therefore, there is the large position of gradient variable in the object region of the downside reducing width.Here, gradient refers to the difference of the height between neighborhood pixels.That is, gradient is less, then this object region is that the possibility of the object being positioned at overlapping upside is higher.According to above content, using gradient as one of evaluation index.Gradient is less, be then that the possibility of the object of upside is higher, Gradient value is set to larger.Figure 11 is the process flow diagram of the example representing Grad computing.
As an example, the overlap of Fig. 8 (a) is considered.Fig. 8 (e) is the image represented according to the order purple from high region to low area, orange, yellow, light blue, blueness.Red dotted line and green dotted line are the object candidates reducing width, and red candidate represents the object being positioned at upside.In the region of the candidate of green, there is the large border of gradient and ladder poor.
Then, as evaluation index 4, the limit goodness of fit of object candidate is described.In the presumption of overlap scheme, do not spend the simple plane template coupling of computing time.The goodness of fit now used is the goodness of fit of the interior zone of candidate.Here, in order to compare the goodness of fit in further detail, the limit of candidate and the goodness of fit at angle are included in evaluation index.Later the goodness of fit at the angle of candidate is described.
In the presumption of overlap scheme, such as, as the candidate of the green dotted line of Fig. 7 (c), there is the possibility that overlapping candidate converges on local minimum.Even converge on the candidate of local minimum, also exist because of the position of convergence and the evaluation of estimate of area, volume, gradient becomes large possibility.Therefore, need to use evaluation index other than the above relatively to reduce to make the evaluation of estimate of this candidate.If candidate converges on the position existing for object, then the profile in object region and the limit of candidate consistent.In addition, for the profile of the object on the upside of being positioned at, high region is also consistent with the border of low area.Figure 12 is the process flow diagram of an example of the computing representing the limit goodness of fit.
As an example, the overlap of Fig. 8 (a) is considered.Fig. 8 (f) represents high region by lavender, represents the image of low area by blueness.Red dotted line and green dotted line are object candidates, and red candidate represents the object being positioned at upside.Orange and yellowish green rectangular portion represents the limit of the candidate consistent with the profile in the profile in object region and high region.The length on the consistent limit of red candidate one side is long.
Then, as evaluation index 5, the angle goodness of fit of object candidate is described.If candidate converges on the position existing for object, then the angle of the profile in object region and the angle of candidate consistent.In addition, the angle being positioned at the Ye Yugao region, angle of the profile of the object of upside is consistent.The minimum of the angle goodness of fit be one, angle all do not mate 0, mxm. be all angles all consistent 4.The coupling goodness of fit at angle is larger, then angle evaluation of estimate of coincideing is larger.
First, from the extraction angle, region existing for object.Figure 13 represents the angle of extracting out from object.Here, the square be made up of solid color represents pixel.In the color of pixel, red expression angle, the frame in the region existing for light blue expression object, black represents the region existing for the object after eliminating frame, angle.The redness that numeral in pixel azury is adjacent with this pixel, black, pixel count azury.According to Figure 13, the pixel (redness) at angle and the pixel count adjacent with angle (light blue, black) N and pixel count (redness, black, the light blue) M adjacent to the pixel (light blue) adjacent with angle
1, M
2there is following relation (to have when two at red pixel and be set to M respectively
1, M
2).According to this relation, extract the angle in the region existing for object out.Figure 14 is the process flow diagram of the example representing angle pump-and-treat system.Figure 15 is the process flow diagram of an example of the goodness of fit computing representing angle.
1.N≤3 and M
1≤ 5 and M
2≤ 6
2.N=4 and M
1≤ 4 and M
2≤ 5
3.N=5 and M
1≤ 4 and M
2≤ 4
As an example, the overlap of Fig. 8 (a) is considered.Fig. 8 (g) represents high region with purple, represents the image of low area by blueness.Red dotted line and green dotted line are object candidates, and red candidate represents the object being positioned at upside.Orange and yellowish green right-angle triangle represents the angle of the candidate consistent with the angle of the profile in the angle of the profile in object region and high region.The quantity at the consistent angle of red candidate one side is many.
Then, the presumption up and down of the overlap that the comprehensive evaluation based on multiple index is carried out is described.When object is soft bag-shaped workpiece, the evaluation index being positioned at the object candidate of top side may not be all always good.Reason is, object creates distortion or transmission light, and acquired range image is coarse.Therefore, if compare each index, then likely the object being positioned at downside is mistaken and erroneous judgement is the object being positioned at top side.Therefore, the whole of These parameters are used to carry out comprehensive descision.
First, be normalized in order to the scope (scale) of the value of each evaluation index unified.But, the angle goodness of fit owing to being weighted so do not do normalized.If by N
cindividual object candidate i (1≤i≤N
c) area value be set to S
i, bulking value is set to V
i, Grad is set to G
i, the limit goodness of fit is set to F
i, the angle goodness of fit is set to CF
i, then area evaluation of estimate E
s, volume assessment value E
v, Gradient value E
g, limit coincide evaluation of estimate E
fand angle coincide evaluation of estimate E
cFbecome formula (6) ~ formula (10) respectively.
E
CF＝CF
i(10)
Next, consider the change degree of the evaluation of estimate of each evaluation index, carry out the normalization of the residual quantity of each evaluation of estimate.And, determine weight simultaneously, be superimposed as comprehensive evaluation value.If the weight of area, volume, gradient, the limit goodness of fit, the angle goodness of fit is set to w respectively
v, w
s, w
g, w
f, w
cF, then comprehensive evaluation value E becomes formula (11).
Then, the experiment of presumption up and down of overlap is described.Object in this experiment is packed macaroni such shown in Figure 16 (a).Prior body plan 30 overlap modes that can expect, obtain its data.To these data, verify the validity of the method proposed.Here, this method has the feature how many results estimated up and down is subject to the impact of the presumption result of overlap scheme, therefore increases the weight of limit, angle goodness of fit evaluation of estimate.Thus, the weight of the comprehensive evaluation used in experiment is w
s=1, w
v=1, w
g=1, w
f=2, w
cF=2.
The process of the process of the present embodiment is represented with the data instance of Figure 16.Figure 16 (b) represents range image.In range image, represent the elevation information of this pixel by color, be changed to redness, yellow, green, light blue, blue continuously according to order from high to low.By the presumption of overlap scheme, obtain (d), (e), (f) and (g) these 4 candidates.For each candidate, employ the value E of the comprehensive evaluation of 5 evaluation indexes
i(i=1 ~ 4) are E
1=22.03, E
2=21.61, E
3=26.47, E
4=22.75, the comprehensive evaluation value of candidate 3 (Figure 16 (f)) is maximum.Thus, as shown in the point in Figure 16 (h), the center of area position of candidate 3 is exported as target jaw position.This Output rusults, at the immediate vicinity of upside object, is therefore picked up successfully.
Figure 17 represents that the immediate vicinity to upside object in prepared 30 data outputs 27 data of target jaw position.In the presumption of overlap scheme, these 27 data are data that candidate and upside object match.On the other hand, in the presumption of overlap scheme, for candidate and unmatched 3 data of upside object, candidate is not mated with upside object.Thus, in the presumption of overlap scheme, if candidate is mated with upside object, then all can determine upside object.Thus, as long as the result of the comprehensive evaluation of carrying out based on multiple index is good, and candidate converges on upside object by pattern match, just can confirm target jaw position reliably export on the upside of the center of object.
Claims (5)
1. a unsetting body position presuming method, to estimating up and down of the overlap of multiple indefinite body, comprising:
Determination step, distance determination unit is to the area test distance that there is described multiple indefinite body;
Threshold value setting procedure, the threshold value that the region that threshold value assigning unit assigns is used for creating the overlap of described multiple indefinite body in the region measured described distance determination unit is distinguished with the region do not overlapped; And
Estimate step up and down, estimate that unit uses described distance determination unit to determine up and down with the distance of described multiple indefinite body, the evaluation index that the part high according to the described threshold value of contrast and/or the part lower than described threshold value carry out evaluating carries out the presumption up and down of the overlap of described multiple indefinite body.
2. unsetting body position according to claim 1 presuming method,
Comprise overlap scheme presumption step, the region that overlap scheme presumption unit measures described distance determination unit, obtain object candidate according to the profile in object region,
Estimate up and down in step described, if there is multiple object candidate, then the described unit that estimates up and down is to described each object candidate, be used in when being positioned at top and become the evaluation index of high praise, be selected to the described object candidate of most high praise as the object being positioned at overlapping top.
3. unsetting body position according to claim 2 presuming method,
Described estimate the region that threshold value is low described in ratio that evaluation index that unit uses comprises described object candidate up and down area, the volume of described object candidate, described object candidate region in gradient, the limit goodness of fit of described object candidate and the angle goodness of fit of described object candidate at least one.
4. a unsetting body position estimating device, to estimating up and down of the overlap of multiple indefinite body, possesses:
Distance determination unit, it is to the area test distance that there is described multiple indefinite body;
Threshold setting unit, it sets the threshold value that the region creating the overlap of described multiple indefinite body in the region for measuring described distance determination unit is distinguished with the region do not overlapped; And
Estimate unit up and down, its use described distance determination unit to determine with the distance of described multiple indefinite body, the evaluation index that the part high according to the described threshold value of contrast and/or the part lower than described threshold value carry out evaluating carries out the presumption up and down of the overlap of described multiple indefinite body.
5. a program, for make computing machine as threshold setting unit with estimate unit up and down and play function,
Described threshold value assigning unit assigns is used for the threshold value distinguished with the region do not overlapped the region that there is in the region of multiple indefinite body the overlap creating described multiple indefinite body being determined distance by distance determination unit,
Described that estimate that unit uses described distance determination unit to determine up and down with the distance of described multiple indefinite body, the evaluation index that the part high according to the described threshold value of contrast and/or the part lower than described threshold value carry out evaluating carries out the presumption up and down of the overlap of described multiple indefinite body.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2014-169814 | 2014-08-22 | ||
| JP2014169814A JP6360747B2 (en) | 2014-08-22 | 2014-08-22 | Amorphous body position estimation method, amorphous body position estimation apparatus and program |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105513040A true CN105513040A (en) | 2016-04-20 |
| CN105513040B CN105513040B (en) | 2020-05-05 |
Family
ID=55635779
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510518788.XA Expired - Fee Related CN105513040B (en) | 2014-08-22 | 2015-08-21 | Method for estimating position of amorphous body, device for estimating position of amorphous body, and program |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP6360747B2 (en) |
| CN (1) | CN105513040B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109955222A (en) * | 2017-12-26 | 2019-07-02 | 发那科株式会社 | Apparatus for handling goods, robot system and article carrying method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000304509A (en) * | 1999-04-21 | 2000-11-02 | Matsushita Electric Works Ltd | Method and device for specifying object |
| JP2010184308A (en) * | 2009-02-10 | 2010-08-26 | Fanuc Ltd | Workpiece taking-out device |
| JP2010210511A (en) * | 2009-03-11 | 2010-09-24 | Honda Motor Co Ltd | Recognition device of three-dimensional positions and attitudes of objects, and method for the same |
| CN102049778A (en) * | 2009-10-26 | 2011-05-11 | 株式会社安川电机 | Robot system and workpiece picking method |
| CN102448679A (en) * | 2009-05-27 | 2012-05-09 | 莱卡地球系统公开股份有限公司 | Method and system for extremely precise positioning of at least one object in the end position in space |
| US20120293623A1 (en) * | 2011-05-17 | 2012-11-22 | Gii Acquisition, Llc Dba General Inspection, Llc | Method and system for inspecting small manufactured objects at a plurality of inspection stations and sorting the inspected objects |
| CN103157607A (en) * | 2011-12-19 | 2013-06-19 | 华南理工大学 | Article identification and sorting device and method thereof |
| CN103310218A (en) * | 2013-05-21 | 2013-09-18 | 常州大学 | Precise recognition method of overlapped shielded fruits |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5322206B2 (en) * | 2008-05-07 | 2013-10-23 | 国立大学法人 香川大学 | Three-dimensional shape measuring method and apparatus |
| JP5332873B2 (en) * | 2009-04-27 | 2013-11-06 | 株式会社Ｉｈｉ | Bag-like workpiece recognition device and method |
| JP5585826B2 (en) * | 2010-06-15 | 2014-09-10 | 株式会社Ｉｈｉ | Object recognition method and apparatus |
-
2014
- 2014-08-22 JP JP2014169814A patent/JP6360747B2/en active Active
-
2015
- 2015-08-21 CN CN201510518788.XA patent/CN105513040B/en not_active Expired - Fee Related
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000304509A (en) * | 1999-04-21 | 2000-11-02 | Matsushita Electric Works Ltd | Method and device for specifying object |
| JP2010184308A (en) * | 2009-02-10 | 2010-08-26 | Fanuc Ltd | Workpiece taking-out device |
| JP2010210511A (en) * | 2009-03-11 | 2010-09-24 | Honda Motor Co Ltd | Recognition device of three-dimensional positions and attitudes of objects, and method for the same |
| CN102448679A (en) * | 2009-05-27 | 2012-05-09 | 莱卡地球系统公开股份有限公司 | Method and system for extremely precise positioning of at least one object in the end position in space |
| CN102049778A (en) * | 2009-10-26 | 2011-05-11 | 株式会社安川电机 | Robot system and workpiece picking method |
| US20120293623A1 (en) * | 2011-05-17 | 2012-11-22 | Gii Acquisition, Llc Dba General Inspection, Llc | Method and system for inspecting small manufactured objects at a plurality of inspection stations and sorting the inspected objects |
| CN103157607A (en) * | 2011-12-19 | 2013-06-19 | 华南理工大学 | Article identification and sorting device and method thereof |
| CN103310218A (en) * | 2013-05-21 | 2013-09-18 | 常州大学 | Precise recognition method of overlapped shielded fruits |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109955222A (en) * | 2017-12-26 | 2019-07-02 | 发那科株式会社 | Apparatus for handling goods, robot system and article carrying method |
Also Published As
| Publication number | Publication date |
|---|---|
| JP6360747B2 (en) | 2018-07-18 |
| JP2016045088A (en) | 2016-04-04 |
| CN105513040B (en) | 2020-05-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5759161B2 (en) | Object recognition device, object recognition method, learning device, learning method, program, and information processing system | |
| CN109523501A (en) | One kind being based on dimensionality reduction and the matched battery open defect detection method of point cloud data | |
| CN105874350B (en) | Calibrating installation, calibration method and calibration procedure | |
| CN105444678B (en) | Handset size measuring method and system | |
| CN103984362B (en) | Position and orientation measurement apparatus, information processor and information processing method | |
| US9111177B2 (en) | Position/orientation measurement apparatus, processing method therefor, and non-transitory computer-readable storage medium | |
| JP5800494B2 (en) | Specific area selection device, specific area selection method, and program | |
| US7627447B2 (en) | Method and apparatus for localizing and mapping the position of a set of points on a digital model | |
| CN100494886C (en) | Three-dimensional scanning system circular index point detection method | |
| US20160061582A1 (en) | Scale estimating method using smart device and gravity data | |
| US20160117824A1 (en) | Posture estimation method and robot | |
| JP4900204B2 (en) | Object recognition method | |
| CN104121902B (en) | Implementation method of indoor robot visual odometer based on Xtion camera | |
| US20180108143A1 (en) | Height measuring system and method | |
| JP2017032548A (en) | Using 3d vision for automated industrial inspection | |
| CN103177444A (en) | Automatic SAR (synthetic-aperture radar) image rectification method | |
| CN101980243A (en) | Binocular vision-based finger vein three-dimensional identification method and device | |
| JP2015015019A (en) | Semi-supervised method for training multiple pattern recognition and registration tool models | |
| CN105335699B (en) | Read-write scene is read and write intelligent identification and the application thereof of element three-dimensional coordinate | |
| CN108513121A (en) | The method and apparatus of depth of field graph evaluation for scene | |
| US20170125271A1 (en) | Position detection apparatus, position detection method, information processing program, and storage medium | |
| US11946986B2 (en) | Magnetic detector, detection method, and non-transitory computer readable storage medium | |
| CN105354828B (en) | Read and write intelligent identification and the application thereof of reading matter three-dimensional coordinate in scene | |
| CN105513040A (en) | Shapeless body position presumption method, and shapeless body position presumption device and program | |
| Lo et al. | Depth estimation based on a single close-up image with volumetric annotations in the wild: A pilot study |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200505 Termination date: 20210821 |