JP2004144719A - Measuring method and instrument of object position/form by ultrasonic array sensor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable to track an object which is moving relatively to a sensor even if occlusion is generated, in a measuring method and an instrument of object position/form by an ultrasonic array sensor. <P>SOLUTION: Position information and reflection intensity of the object to be tracked are detected by using the ultrasonic sensor, and a detecting object map ZM and a reflection intensity map RM are obtained. By simulating an intensity value at each point by using the maximum reflection intensity value on the front side of the sensor as a standard, data only in a range which is predetermined based on an upper limit H and a lower limit L of the reflection intensity are extracted and a reliable and new detection object map ZM is created. Object tracking is executed with respect to a tracking point group on the detection object map 2M obtained in a time series order while performing decision of characteristic point, detection of occlusion, and inclusion of occlusion to a pattern for matching. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method and apparatus for measuring the position and shape of an object using an ultrasonic array sensor.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as a sensor for measuring the position and shape of a measurement target, an array-type ultrasonic sensor that receives reflected ultrasonic waves from the measurement target is known. In this sensor, ultrasonic receiving elements are arranged in an array, and detect the distance to an object based on the time from emission of ultrasonic waves to reception of reflected waves. Ultrasonic waves have a slower propagation speed than light, so distance measurement is easy. Also, instead of mechanically oscillating the ultrasonic sensor and scanning in the measurement direction, electronic scanning can be performed by performing delay processing on the output of the receiving elements arranged in an array. it can. As such an array-type ultrasonic sensor, an array-type sensor characterized by a configuration aiming at miniaturization (for example, see Patent Literature 1), or an ultrasonic-wave transmission unit is array-type and is transmitted from each transmission element. There is an array type sensor that is used in combination with an ultrasonic wave in which the phase of an ultrasonic wave is shifted to control the direction of a main beam formed by the interference phenomenon of a wave (for example, see Patent Document 2).
[0003]
[Patent Document 1]
JP-A-2002-156451
[Patent Document 2]
JP 2001-305222 A
[0004]
[Problems to be solved by the invention]
However, in Patent Documents 1 and 2 described above, there is no mention of an occlusion problem in tracking an object that moves relatively to a sensor and processing of ambiguous data due to noise or the like. Such a point is not a problem when the target object is stationary, but becomes a problem particularly when the moving object is tracked. An ultrasonic sensor in which ultrasonic receiving elements are arranged in an array electronically scans the direction of the object, and tracks while moving the object relative to the sensor while measuring the surface position and shape facing the sensor In this case, the occlusion which is not measured may occur due to the shadow of the target object itself depending on the positional relationship between the unevenness of the surface of the target object and the sensor. When such occlusion occurs, the apparent surface shape of the target object to be measured changes, making it difficult to track the target object. When ambiguous data such as noise is included, tracking becomes more difficult.
[0005]
An object of the present invention is to solve the above-mentioned problem, and to measure an object position / shape using an ultrasonic array sensor and to measure the object, which can track an object moving relative to the sensor even if occlusion occurs. It is intended to provide a device.
[0006]
Means for Solving the Problems and Effects of the Invention
In order to achieve the above object, the invention according to claim 1 uses a position detection data of an object detected by an ultrasonic array sensor to detect a surface position of an object that moves relatively to the sensor and / or a surface position of the object that moves relative to the sensor. Or, in the method of measuring the position and shape of an object for measuring the surface shape, a map creating step of expanding the measured surface position and surface shape data on a detected object map, and the reflection intensity at each point on the detected object map. An intensity map creating step of expanding the data into a reflection intensity map, and reliability of data that changes substantially smoothly within a predetermined range temporally or spatially for each of the data shown in the detected object map and the reflection intensity map. A data extraction step of extracting the data as having certain data and creating a new detected object map.
[0007]
In the measurement method, using an ultrasonic sensor in which ultrasonic receiving elements are arranged in an array, the relative position between the sensor and the object is detected in a time series to detect position information and reflection intensity of the object, and these are detected. Since the information is combined, defective data such as noise is identified and discarded, reliable surface position and surface shape data can be obtained. Obtaining such data facilitates and reliably tracks an object that moves relative to the sensor.
[0008]
According to a second aspect of the present invention, in the method for measuring the position and shape of an object according to the first aspect, the detection data of the surface position and / or the surface shape of the relatively moving object extracted at the data extracting step is at least two times. By comparing the occlusion estimation step of estimating the occurrence of occlusion of the detected object, and correcting the matching pattern used for tracking the object to the position and orientation of the object when the occlusion occurs, and incorporating the occlusion. The method further includes a pattern generating step of generating a new matching pattern, and a tracking step of performing the next pattern matching using the generated new matching pattern to track the position and / or orientation of the object. .
[0009]
In the above measurement method, reliable detection data of the surface position and the surface shape are obtained two or more times under different conditions of the relative position, and these are compared, so that the occlusion estimation and the updating of the matching pattern can be performed with high accuracy. Yes, even if occlusion occurs, the target object can be tracked.
[0010]
According to a third aspect of the present invention, in the object position / shape measuring method according to the first or second aspect, the detected object maps extracted in the data extracting step are obtained in a time series, and each of the detected object maps is acquired. A predetermined feature point is determined as a representative point of the group from the group of the measurement points of the object, and for each of the groups preceding and succeeding in time series, the representative point of the preceding group and the representative of the subsequent group corresponding to this representative point are set. When the moving distance from the point is equal to or less than a predetermined amount, a tracking step of determining that the previous group and the subsequent group correspond to the same group of the object and tracking the position and / or orientation of the object is performed. It has more.
[0011]
In the above measurement method, attention is paid to representative points, which are characteristic points of the tracking point group, and tracking is performed by determining identity based on the moving distance of each representative point. Correspondence can be reduced. Further, since the judgment is made based on the representative points, the judgment processing time is reduced.
[0012]
According to a fourth aspect of the present invention, in the object position / shape measuring method according to any one of the first to third aspects, the detected object map extracted in the data extracting step is obtained in time series. For the detected object map, a predetermined feature point is determined as a representative point of the group from the group consisting of the measurement points of the object, and for each group preceding and succeeding in time series, the reflection intensity of the representative point of the preceding group and the representative point If the intensity difference between the corresponding group and the reflection intensity of the representative point in the subsequent group is equal to or less than a predetermined amount, it is determined that the previous group and the subsequent group correspond to the same measurement point group of the object, and the object is determined. Further comprising a tracking step of tracking the position and / or attitude of the camera.
[0013]
In the above measurement method, attention is paid to representative points, which are characteristic points of the tracking point group, and tracking is performed by determining identity based on the reflection intensity of each representative point. Correspondence can be reduced. Further, since the judgment is made based on the representative points, the judgment processing time is reduced.
[0014]
According to a fifth aspect of the present invention, in the method for measuring the position and shape of an object according to any one of the first to fourth aspects, in the intensity map creating step, a measurement point whose reflection intensity is equal to or more than a predetermined value is effectively measured in the object map. Value.
[0015]
In the above-described measurement method, only the data having a high reflection intensity of the received reflected ultrasonic waves is used as an effective value, so that a highly reliable measurement value can be obtained. Obtaining such measured values makes it easy and reliable to track an object that moves relative to the sensor.
[0016]
According to a sixth aspect of the present invention, in the object position / shape measuring method according to the first aspect, the detected object map extracted in the data extracting step is obtained in a time series, and the detected object ahead of and behind the time series is obtained. Performs geometric transformation including movement and rotation using a group consisting of the measurement points of the object on the map as a matching pattern, and superimposes this on a group in the subsequent detected object map to detect the target whose similarity is greater than or equal to or higher than a predetermined value. Then, an occlusion detection step of detecting an occlusion occurrence portion based on the detected target, and generating a new matching pattern corrected by incorporating the occlusion into the matching pattern, and generating the generated matching pattern. Tracking step of tracking the position and / or orientation of an object by performing the following pattern matching using , In which further comprises a.
[0017]
In the above measurement method, a corresponding point in the preceding and following object maps is detected, an occlusion is detected by a matching pattern generated based on the detected point, an occlusion is incorporated, a matching pattern is generated again, and this refreshed pattern is obtained. Since the tracking is performed using the matching pattern, even if the measured apparent surface shape of the target object changes due to the occurrence of occlusion, the target object can be tracked by pattern matching.
[0018]
According to a seventh aspect of the present invention, in the object position / shape measuring method according to the sixth aspect, in the occlusion detecting step, a portion where occlusion occurs is estimated based on shape data of the object known in advance, and the geometric transformation is performed. Is corrected by the estimated occlusion occurrence portion to generate a new matching pattern.
[0019]
In the above-described measurement method, even if the measured apparent surface shape of the target object changes due to occlusion, occlusion is more reliably incorporated, and tracking of the target object by pattern matching becomes easier and more reliable.
[0020]
According to an eighth aspect of the present invention, in the method for measuring an object position / shape according to the sixth or seventh aspect, a matching pattern having a large curvature portion of a curve formed by a continuous point sequence in a detected object map as a feature point. Is included.
[0021]
In the above-described measurement method, a portion having a large curvature, which is a feature point of the tracking point group, is focused on as a representative point, and tracking is performed by judging identity based on the moving distance of each representative point. It is possible to reduce erroneous correspondence of tracking points between object maps. In addition, options for selecting a representative point for tracking are expanded, and tracking can be performed more reliably.
[0022]
According to a ninth aspect of the present invention, in the object position / shape measuring method according to any one of the sixth to eighth aspects, a curvature of a point sequence curve included in a group including measurement points of the object in the detected object map is: A tracking point identification step of determining that the previous group and the subsequent group correspond to the same measurement point group of the object when the plurality of detected object maps show a change equal to or less than a predetermined amount, Have
[0023]
In the above measurement method, focusing on the curvature of the tracking point sequence, tracking is performed by judging the identity based on the amount of change, so that the time series detection object map of the shape object with few feature points such as corners is used. It can be tracked and wrong responses can be reduced. In addition, options for selecting a representative point for tracking are expanded, and tracking can be performed more reliably.
[0024]
According to a tenth aspect of the present invention, in the method for measuring an object position / shape according to any one of the sixth to ninth aspects, the first and second groups of measurement points of the object on the detected object map are provided. When a geometrical transformation is performed on the first group in the earlier detected object map in chronological order to obtain a group corresponding to the first group in the later detected object map, the same transformation is performed on the first detected object map. When the distance between the converted group point and the subsequent point of the second group in the detected object map is equal to or less than a predetermined value, the first and second groups are formed. The method further includes a regrouping step of forming one group.
[0025]
In the above measurement method, the same geometric transformation is performed for two tracking point groups, and the same group property of the two groups is determined based on the behavior of the tracking points for the transformation. Can be identified, and object tracking can be performed more reliably.
[0026]
According to an eleventh aspect of the present invention, in the method for measuring the position and shape of an object according to any one of the sixth to tenth aspects, a trajectory obtained by tracking measurement points of a group of temporally continuous detected object maps is used. The method further includes a trajectory prediction step of estimating the position of the group in the next detected object map.
[0027]
In the above measurement method, the next movement position is estimated based on the past movement trajectory of the tracking point, so that the calculation time required for tracking in the next detected object map can be reduced.
[0028]
According to a twelfth aspect of the present invention, in the object position / shape measuring method according to any one of the sixth to eleventh aspects, a slope of a regression line obtained from a point sequence in a group including measurement points of the object in the detected object map. However, when a plurality of the detected object maps change substantially smoothly, it is determined that the group of the detected object map and the group of the detected object map correspond to the same measurement point group of the object. It further includes a straight line tracking step.
[0029]
In the above measurement method, focusing on the inclination of the regression line of the tracking point sequence, tracking is performed by judging identity based on the amount of change, so that a time-series detected object of a shape object with few feature points such as corners is obtained. Tracking between maps can be performed, and incorrect responses can be reduced. In addition, options for selecting a representative point for tracking are expanded, and tracking can be performed more reliably.
[0030]
According to a thirteenth aspect of the present invention, an ultrasonic sound source, ultrasonic transmission control means for controlling ultrasonic transmission of the ultrasonic sound source, and detecting an ultrasonic wave transmitted from the ultrasonic sound source reflected by a detection object. An ultrasonic array sensor, an ultrasonic signal receiving means for receiving an ultrasonic signal detected by the ultrasonic array sensor in synchronization with the ultrasonic transmission control means, and each element output of the ultrasonic array sensor from the ultrasonic signal receiving means Object position detecting means for processing the signal to detect the position of the detected object; and generating the surface position and surface shape data of the detected object from the position detection data detected by the object position detecting means, and generating the generated surface position and surface shape data. Map creation function means for developing the object map, the intensity map creation function means for developing the reflection intensity data at each point on the object map into a reflection intensity map, For each data shown in the detected object map and the reflection intensity map, data that changes substantially smoothly within a predetermined range temporally or spatially is extracted as reliable data, and a new detected object map is extracted. An object position / shape measuring device comprising: an object position / shape measuring means for measuring a position / shape of a detected object using a data extracting function means to be created.
[0031]
In the above configuration, an operation similar to any one of the first to twelfth aspects can be obtained. In addition, a simple device for measuring the surface position and shape of a relatively moving object and tracking the object using the ultrasonic array sensor can be obtained.
[0032]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a method and apparatus for measuring the position and shape of an object using an ultrasonic array sensor according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a positional relationship between an ultrasonic array sensor (hereinafter, sensor) and an object. Here, the objects 1 and 2 and the sensor S are moving relative to each other, and the sensor S moves from the point P1 to the point P2 and measures. The sensor S includes a receiving surface including a plurality of ultrasonic receiving elements arranged two-dimensionally, a sound source that emits ultrasonic waves, and a control unit that controls transmission and reception of ultrasonic waves and performs signal processing. The sensor S emits an ultrasonic wave from a sound source, and receives the ultrasonic waves reflected on the surfaces of the three-dimensional objects 1 and 2 by a plurality of elements of a receiving unit. Based on the time from when the ultrasonic wave is transmitted from the sound source to when the ultrasonic wave is received by the sensor S, the distance between the sensor S and each part of the object surface where the ultrasonic wave is reflected is measured. At the same time as measuring the distance, the intensity of the received reflected ultrasonic wave is also measured. By electrically scanning the signal received by each receiving element, the sensor S can measure the distance and the direction, for example, at the point P1, the horizontal direction X1, the depth or the distance Z1, and the vertical position Y1. . The shape of the object surface is expressed as discrete point cloud data.
[0033]
FIG. 2 shows an example of surface position / shape data measured at a certain time. FIGS. 3A and 3B show a detection situation and a detected object map. Here, the measurement results at specific height positions Y1 = y1, Y2 = y2 (y1 = y2) are shown, and corresponding to the measurement points m1, m2 on the object surface facing the sensor S, the detected object map ZM1, It is represented in ZM2 as a point cloud. FIG. 4 shows the plurality of detected object maps ZM1 and ZM2 obtained as described above on the same map from the viewpoint of the sensor S.
[0034]
FIGS. 5A and 5B show a detected object map and a reflection intensity map, respectively. Data extraction for improving reliability is performed on the detected object map data using the reflection intensity map data as follows. First, it is assumed that a detection object map ZM including a series of measurement point data 11 to 15 and a measurement point 16 shown in FIG. At this time, the intensity of the received ultrasonic wave corresponding to each point on the detected object map ZM is developed as a reflection intensity map RM shown in FIG. On the developed reflection intensity map RM, the reflection intensity data is distributed as a series of data 21 to 25 and 26. Depending on the distance from the sensor S and the deflection angle from the front position of the sensor S, the closer to the sensor S and the closer to the front of the sensor S, the more the reflection intensity is distributed. Therefore, the reflection intensity data is grouped and distributed. are doing.
Generally, ultrasonic waves are attenuated with propagation, and the reflection intensity changes depending on the angle between the receiving surface of the sensor S and the reflector. Therefore, the upper limit H and lower limit L of the reflection intensity are obtained by simulating the intensity value at each point based on the maximum reflection intensity value at the front of the sensor, and based on these upper limit H and lower limit L, It is possible to select only data within a predetermined range defined as reliable data and discard others. For example, the point 16 in the detected object map ZM is discarded as noise because the corresponding point 26 in the reflection intensity map RM is out of the predetermined range.
[0035]
FIG. 6 shows a processing flow for performing the above-described reliability data extraction in a series of measurements. First, an ultrasonic wave is transmitted toward the object, and the ultrasonic wave reflected from the object is received by the ultrasonic array sensor (S101, S102). A delay operation (electronic scanning) is performed on a signal received by each receiving element constituting the ultrasonic array sensor, and the surface position / shape data of the object present in each direction in the field of view of the sensor and the reflection intensity of the ultrasonic wave. Data is created (S103). Next, these data are developed into a detected object map ZM (n) and a reflection intensity map RM (n) on a specific horizontal plane (S104). Next, each data in the reflection intensity map RM (n) is determined by a determination condition based on the ultrasonic wave propagation condition J, reliable data is extracted, and other data is discarded. The ultrasonic wave propagation condition J includes a medium condition in the ultrasonic wave propagation path, an ultrasonic wave transmission condition, a reflection condition, a reception condition, and the like. Further, each data of the detected object map ZM (n) is extracted or discarded with extraction or discarding of each data of the corresponding reflection intensity map RM (n) (S105). Based on the extracted data, a reliable detected object map ZM (n) is re-created. Based on this, the object position / shape is determined, and the n-th measurement is completed (S106, S107). Thereafter, the (n + 1) -th measurement is performed in the same manner as described above (S108).
[0036]
Next, occlusion estimation and moving object tracking will be described. FIG. 7 shows how occlusion occurs. Occlusion occurs when a surface portion that cannot be seen from the sensor S is formed behind the measured object itself. The reflected ultrasonic wave RU does not enter the sensor S from the occlusion occurrence point OC in the figure. When such occlusion occurs, the measured apparent shape of the target object changes, which makes it difficult to track the target object. In order to solve this problem, the position / shape detection data of the object is compared at least twice or more, an occurrence portion of the occlusion is estimated, and the matching pattern is corrected so as to become the occlusion at the position and orientation. The position and orientation of the target object are tracked by performing pattern matching using the occlusion correction pattern for matching.
[0037]
FIG. 8 shows a processing flow for performing position / shape measurement and object tracking in consideration of the occlusion correction as described above. First, an object map ZM (n) obtained by performing the n-th measurement and extracting the above-described reliable data is obtained (S201). Next, a pattern for detecting a target tracking object on the detected object map ZM (n) using the matching pattern M (n) obtained from the previous measurement data or data obtained by prediction in advance. Matching is performed (S202). When the pattern matching fails, that is, when the target object cannot be detected (No in S203), the structure of a group of points to be tracked (tracking point group) on the detected object map ZM (n) changes due to the occurrence of occlusion. In the next step, the occlusion is estimated and the matching pattern is corrected. For occlusion estimation, k (k ≧ 2) time-series detected object maps ZM (i), i = n1 to nk, in which the relative positions of the object and the sensor are different (S204). The time series detection map ZM (i) is acquired at a relative position change interval necessary and sufficient to track the occurrence of occlusion. Depending on the speed of change of the relative position and the like, acquisition is performed first at coarse intervals and then at finer intervals. Pattern matching is performed on the obtained corner detection object map ZM (i) to estimate occlusion (S205), and the obtained occlusion information is incorporated into a matching pattern M (n) to perform new matching. The pattern M (n) is used (S206). Thereafter, normal tracking is continued (S207, S208). Also, when the pattern matching is successful in step S203 and the target object is detected (yes in S203), the tracking is continued to the measurement in the state where the relative position has changed (S207, S208).
[0038]
Next, a method of tracking a group between time-series consecutive detected object maps will be described. FIG. 9 shows two tracking point groups G (n-1) and G (n) which are ahead and behind in time series on the detected object map. In the point sequence of each tracking point group, a plurality of characteristic points a (j, k) such as an end point and a vertex of a polygonal line are determined as representative points of the group. Then, the absolute values of the corresponding points of the representative points, for example, a vector V3 connecting a (n-1,3) and a (n, 3), and similarly V2 and V1, are obtained. Is less than or equal to a predetermined determination value, it is determined that the tracking point group G (n-1) and the group G (n) correspond to the same point group on the object, and tracking is performed as the same group.
[0039]
FIG. 10 shows a processing flow of the above-described tracking method. First, the (n-1) th detection object map creation and the tracking point group G (n-1) are determined (S301), and m features are obtained from the point group in the tracking point group G (n-1). A point is determined (S302).
Next, steps similar to the above are performed for the n-th measurement with the relative position between the object and the sensor changed (S303 to S305). Next, the moving distance of each representative point is obtained by calculating the absolute value of the moving vector between each point, for example, | a (n-1) -a (n) |, and the moving distance between the value and a predetermined determination value is calculated. The shift D (i) is calculated (S306). Next, when all the deviations D (i) are positive, that is, when the movement amount is smaller than the determination value (yes in S307), the tracking point groups G (n-1) and G (n) become the same point group on the object. It is determined that they correspond (S308). In this case, the tracking has been successful, and the tracking is continued (S309). If no in step S307, it is determined that tracking has failed, and exception processing is performed (S310). In the above description, the moving distance of each point is compared with a predetermined determination value. However, the variation between the moving distances of each point is compared with a predetermined determination value, and when the variation is equal to or less than the determination value, the tracking point group G ( It may be determined that n-1) and the group G (n) correspond to the same point group on the object.
[0040]
Next, another method of tracking a group between successively detected object maps in chronological order will be described. FIG. 11 shows two tracking point groups G (n-1) and G (n) preceding and succeeding in time series on the detected object map. In each of the tracking point groups G (n-1) and G (n), a plurality of feature points a (j, k) such as an end point and a vertex of a polygonal line are determined as representative points of the group. Then, the reflection intensities r (n-1,3) and r (n, 3) at the corresponding points of the representative points, for example, a (n-1,3) and a (n, 3) are compared. When the difference between the reflection intensities for each corresponding point is equal to or less than a predetermined amount, it is determined that the tracking point group G (n-1) and the group G (n) correspond to the same point group on the object, and are tracked as the same group. Is done.
[0041]
FIG. 12 shows a processing flow of the above-described tracking method. The processing flow in this case uses the reflection intensity r (j, k) instead of the representative point a (j, k) in FIG. 10 described above, and uses the judgment value instead of the deviation D (i) from the judgment value. The difference is that E (i) = (judgment value) − | r (n, i) −r (n, i) | As another tracking method, the tracking method using the moving distance shown in FIGS. 9 and 10 may be combined with the tracking method using the difference between the reflection intensities shown in FIGS. 11 and 12. When both the distance and the difference between the reflection intensities are equal to or smaller than the respective predetermined amounts, it is determined that the tracking point group G (n-1) and the tracking point group G (n) correspond to the same point group on the object. Can be tracked reliably.
[0042]
Next, a method for improving the reliability of the detected object map by setting a lower limit for the reflection intensity will be described. FIG. 13 shows a processing flow of the method. At each measurement point, if the reflection intensity is equal to or more than a predetermined amount, the possibility of noise is low, and since the S / N as a signal is large, the reliability of the calculation result of the direction and the distance increases. First, an ultrasonic wave is transmitted toward an object, and the ultrasonic wave reflected from the object is received by the ultrasonic array sensor (S501, S502). A delay operation (electronic scanning) is performed on a signal received by each receiving element constituting the ultrasonic array sensor, and the surface position / shape data of the object present in each direction in the field of view of the sensor and the reflection intensity of the ultrasonic wave. Data is created (S503). Next, each reflection intensity value is compared with a predetermined judgment value, and when it is larger than the judgment value (yes in S504), the process proceeds to the next step, and when it is smaller than the judgment value (no in S504), the reflection intensity is determined. The surface position / shape data and reflection intensity data corresponding to the data are discarded (S505). After this processing has been performed for all the measurement data (yes in S506), the remaining data that has not been discarded is developed into the detected object map ZM (n) and the reflection intensity map RM (n) (S507). Next, the process of extracting reliable data described in FIG. 6 is performed, and the object position / shape is determined based on the finally obtained detected object map ZM (n), and the n-th measurement is completed. (S508 to S510). Thereafter, the (n + 1) -th measurement is performed in the same manner as described above (S511).
[0043]
Next, a method of correcting occlusion and a tracking method by pattern matching will be described. FIG. 14 shows two tracking point groups that come and go in time. The preceding tracking point group G (n-1) is measured from almost the front with respect to the convex portion on the object surface, and the following tracking point group G (n) is measured from the diagonal right direction of the convex portion. I have. Therefore, the occlusion OC has occurred in the tracking point group G (n). When occlusion is included in the measurement data, measurement is continuously performed twice or more. That is, the matching pattern can be corrected using the detected object map obtained by measuring the object that is moving relatively from different directions.
[0044]
FIG. 15A shows how the above-described tracking point group is subjected to geometric transformation to detect occlusion. A tracking point group G (n-1) that does not include occlusion is used as a matching pattern, and a geometric transformation including rotation and translation is performed on the group. Do. This is because, although not shown on the detected object map, there are usually a plurality of tracking point groups G (n), and it is necessary to detect a group corresponding to the group being tracked from among them. When a corresponding tracking point group can be found, the parallelism and the like of the measurement point sequence constituting the tracking point group are compared, and a portion that is significantly different from the shape of the tracking point group G (n) in the matching pattern is generated. Extract as a part.
[0045]
FIGS. 15B and 15C show a method of generating a new matching pattern. The position of the matching pattern is corrected so as to correspond to the position and orientation, and the occlusion correction is performed based on the extracted part information to generate a new matching pattern. Thereafter, pattern matching is performed using the corrected matching pattern. Thus, even if occlusion occurs in the measured target object and the apparent surface shape changes, the target object can be tracked by pattern matching.
[0046]
FIG. 16 shows a processing flow of the above-described occlusion correction. First, since an ultrasonic wave is transmitted toward an object, creation of an object map ZM (1) and determination of the object position and shape are performed (S601, S602), and object measurement points on the object map ZM (1). , A tracking point group G (1) is determined (S603). A matching pattern M (1) is generated based on the tracking point group (1) (S604). The matching pattern M (1) is used for pattern matching in the next measurement and object tracking. Similarly, the n-th measurement is performed, and the tracking point group G (n) is determined (S605 to S607). At this point, assuming that the tracking point group G (n) includes occlusion, a method of generating a matching pattern M (n) for the tracking point group G (n) including such occlusion will be described. I do. First, a conversion tracking point group K obtained by subjecting a matching pattern M (n-1) set based on the previously measured tracking point group G (n-1) to geometric conversion including translation and rotation. (N-1) is obtained (S608). Next, the similarity between the tracking point group G (n) and the conversion tracking point group K (n-1) is maximized while changing the condition of the geometric conversion (S609, S610). A matching pattern M (n) is generated based on the conversion tracking point group K (n-1) when the similarity is maximized (S611). Next, each point of the tracking point group G (n) is compared with each point of the generated matching pattern M (n) to detect an occlusion occurrence portion (S612 to S614). Next, a new matching pattern M (n) corrected to incorporate occlusion into the matching pattern M (n) is generated (S615). In the next measurement and tracking, pattern matching is performed using the corrected matching pattern M (n).
[0047]
FIG. 17 shows a method of generating a matching pattern when the site where occlusion occurs is known in advance. If the shape of the target object is known, a change in the appearance of the target object, and thus the occurrence of occlusion, can be predicted with respect to the geometric transformation performed during pattern matching. Therefore, the matching pattern can be easily corrected. In addition, when the shape of the target object and the movement of the target object such as rotation are known in advance, the correction of the matching pattern becomes easy similarly.
Such prior information is particularly effective in tracking when an intervening object moves between the sensor and the target object. By performing occlusion correction on the matching pattern, even if the apparent surface shape of the measured target object changes due to occlusion, the target object can be tracked by pattern matching.
[0048]
Next, determination of a representative point focusing on the curvature of a curve formed by a series of measurement points in the detected object map will be described. FIG. 18 shows a point sequence curve on the detected object map. In the tracking point group on the detected object map, if the curved line has no vertex or the like of the polygonal line composed of the measurement point sequence, the characteristic point is only the end point of the point sequence, and the tracking reliability is reduced. In such a case, a portion where the amount of change in curvature is large is determined, and by using these points p1, p2, and p3 as feature points, erroneous correspondence between the time-series tracking point groups can be reduced.
[0049]
Next, a tracking method (tracking point identification method) focusing on the radius of curvature of a curve formed by a series of measurement points in the detected object map will be described. FIG. 19 shows two tracking point groups G (n-1) and G (n) forming a time series on the detected object map. In each group, in the case of a smooth curve having no characteristic point such as a vertex of a polygonal line in the measurement point sequence, the characteristic point is only the end point of the point sequence, and the tracking reliability is reduced. In such a case, the curvature of the curve is obtained, and the radius of curvature can be used instead of the feature point.
[0050]
FIG. 20 shows a processing flow of the above tracking method. First, the radii of curvature ρ (n−1) and ρ (n) are obtained (S701, S702), and the difference | ρ (n−1) −ρ (n) | between them is compared with a predetermined determination value. If the difference between the radii of curvature is equal to or smaller than the predetermined amount (yes in S703), the points of both tracking point groups G (n-1) and G (n) are identified as the same tracking point on the object (S704). Otherwise (no in S703), the points of the tracking point groups G (n-1) and G (n) are determined to be different groups on the object, that is, not tracking target points (S705). As described above, by using the radius of curvature, the tracking points can be identified and tracked, and erroneous correspondence between the time-series tracking point groups can be reduced.
[0051]
Next, a method for determining the identity of a tracking point group and regrouping will be described. FIG. 21 shows a case where there are two tracking point groups at each preceding measurement point. Consider two tracking point groups G1 (n-1) and G2 (n-1) on the detected object map at the same measurement, and two tracking point groups G1 (n) and G2 (n) at another measurement. . The tracking point groups G1 (n-1) and G1 (n) are tracked, and after the tracking is successful, the identity of the tracking point groups G2 (n-1) and G2 (n) is determined.
[0052]
FIG. 22 shows a processing flow of the regrouping method described above. First, the tracking point groups G1 (n-1), G2 (n-1), G1 (n) and G2 (n) at the (n-1) th and nth times are determined (S801 to S803). A geometrical transformation is performed on the tracking point group G1 (n-1) to form a transformation group H1 (n-1) (S804). Next, the similarity determination between the tracking point group G1 (n) and the transformation group H1 (n-1) is performed (S805), and the final geometric transformation T maximizes the similarity, and the tracking point group G1 (n-1) ) And G1 (n) are determined to correspond to the same tracking point (S806 to S808). Next, the final geometric transformation T is performed on the tracking point group G2 (n-1) to form a transformation group H2 (n-1) (S809). Next, a difference D (i) between each point a2 (n, i) in the tracking point group G2 (n) and each point b2 (n-1, i) in the conversion group H2 (n-1) is calculated. (S810). The difference D (i) is compared with a predetermined determination value, and if all D (i) are equal to or less than the determination value (yes in S811), the points of both tracking point groups G1 and G2 should belong to the same group. , And are regrouped into one (S812). Otherwise (no in S811), the points of the tracking point groups G1 and G2 are determined to belong to another group (S813). At this time, the points in another group are considered to be noise or measurement points belonging to another object.
[0053]
Next, a method of estimating a moving area of a tracking point group will be described. FIG. 23 shows three tracking point groups in a time series on the detected object map. In the first two tracking point groups G (n-1) and G (n), a plurality of feature points including the end points of the point sequence in the group and the vertices of the polygonal line are set as the representative points of the group. From the motion vector trajectories V1, V2, V3 connecting the corresponding points between the tracking point groups, the motion vector trajectories W1, W2, W3 and the candidate points estimated to move the tracking point group in the next (n + 1) th measurement To predict. By setting the area around the candidate point as the candidate area for the next pattern matching based on the prediction, the tracking position in the next detected object map can be estimated, so that the calculation time can be reduced.
[0054]
Next, another method for characterizing the tracking point group will be described. FIG. 24 shows two tracking point groups G (n-1) and G (n) in a time series on the detected object map. A plurality of feature points including the end points of the point sequence in each group, the vertices of the polygonal line, and the like are set as representative points of the group. In each group, when the point sequence of these feature points has, for example, two regression lines each having a correlation coefficient equal to or more than a predetermined value, the straight lines L1 (n-1) and L2 (n-n) in G (n-1) are obtained. 1), straight lines L1 (n-1), L2 (n-1) and the like in G (n) are introduced. Then, when the difference between the inclinations of the straight lines corresponding to the tracking is equal to or less than a predetermined amount, the straight lines are grouped together. As a result, even a tracking point group of a measurement point sequence having a shape with a small number of feature points can be tracked in this way, and erroneous correspondence can be reduced. Here, the regression line of one group may be determined to be one or two or more.
[0055]
Next, an object position / shape measuring device incorporating the above-described object position / shape measuring method will be described. FIG. 25 shows a block diagram of the present measuring device. The measuring device detects an ultrasonic sound source 100, an ultrasonic transmission control unit 200 that controls ultrasonic transmission of the ultrasonic sound source 100, and an ultrasonic wave transmitted from the ultrasonic sound source 100 reflected by a detection object. An ultrasonic array sensor 300 to perform the operation, and an ultrasonic signal receiving unit 400 that receives an ultrasonic signal detected by the ultrasonic array sensor 300 in synchronization with the ultrasonic transmission control unit 200. Further, the measuring device processes each element output signal from the ultrasonic signal receiving means 400 to detect the position of the detected object, using the object position detecting means 500 and the position detection data detected by the object position detecting means 500. An object position / shape measuring means 600 for measuring the position / shape of the detected object is provided.
[0056]
The above-described ultrasonic array sensor 300 is an array sensor in which the ultrasonic receiving elements 301 are arranged in, for example, N columns and M rows. The ultrasonic sound source 100 is installed near the ultrasonic array sensor 300 so as to reduce the angle between the transmission direction and the reception direction of the ultrasonic wave. Further, the ultrasonic transmission control means 200 controls the ultrasonic sound source 100 to transmit ultrasonic pulses, and sends a pulse transmission timing signal to the ultrasonic receiving means 400. The ultrasonic receiving means 400 receives the ultrasonic signal received by each ultrasonic receiving element 301 in synchronization with the timing signal. A signal from each ultrasonic receiving element 301 and a signal including time information are sent to the object position detecting means 500.
[0057]
The object position detecting unit 500 electrically scans the signal including the time information received by each of the receiving elements 301 in the delay adding unit 501 according to the arrangement of the receiving elements 301 arranged in an array, The ultrasonic array sensor 300 obtains direction data indicating the direction of each point on the surface of the object facing the object and intensity data in the direction, and sends them to the calculation result storage unit 503. Also, the delay adder 501 sends the received ultrasonic signal reconstructed as a signal for each direction and its time information to the distance calculator 502. The distance calculation unit 502 calculates the distance between the sensor and each point on the surface of the object based on the signal and the time information, and sends the distance to the calculation result storage unit 503. The calculation result storage unit 503 stores and stores these direction data, intensity data, and distance data as position detection data, and sends the data to the object position / shape measurement unit 600.
[0058]
The above-described position detection data is obtained by the detected object map creation unit 601 and the reflection intensity map creation unit 602 of the object position / shape measurement unit 600 for the distance to the horizontal scanning position and the reflection intensity for the same horizontal scanning position for a specific vertical position. It is developed as a distribution map. The data is represented as discrete data on each map. The distance data developed on the detected object map is selected by the data extraction unit 603 by a predetermined determination process based on the data on the reflection intensity map and the propagation and measurement conditions of the ultrasonic wave, and the reliable distance data is obtained. The detected object map obtained by extracting only the object is reconstructed. By performing such a detection object map for all the vertical positions, three-dimensional position data of the object surface facing the ultrasonic array sensor 300 is obtained, and thus the measurement of the object position / shape is performed.
[0059]
Since the distance data of the reconstructed detected object map is the position / shape data of the three-dimensional object, occlusion occurs in the measured data depending on the shape and the measurement direction of the object. With respect to occlusion, the shape of the object is estimated by obtaining measurement data accompanying a change in the measurement position due to the relative movement of the object and the sensor. When the measurement target object is moving, measurement with different relative positions can be performed by performing measurement over time. In addition, by providing a moving device in the object position / shape measuring device main body or the ultrasonic array sensor 300, measurement with different relative positions can be performed. Conversely, when the relative position between the ultrasonic array sensor 300 and the object changes, object tracking can be performed while estimating occlusion using a pattern matching technique.
[0060]
Such occlusion estimation or detection, generation of a pattern for pattern matching and update of the pattern, and tracking of an object are performed by the occlusion detection unit 604, the occlusion estimation unit 607, the pattern generation units 605, 608, and the tracking units 606, 609. This is performed by a series of processes. The result of the occlusion detection or estimation, the matching pattern, the measurement data of the measurement target, and the like are output to the measurement result output unit 610 and stored in the measurement data storage unit 611. The data stored in the measurement data storage unit 611 is used as pattern matching data or occlusion estimation / detection data in the next measurement in which the relative position between the measurement target object and the ultrasonic array sensor 300 has changed. Can be used in the processing in.
[0061]
The present invention is not limited to the above description, and various modifications can be made. For example, the detection object map on one horizontal plane has been described. However, the invention is not limited to this, and the invention can be similarly applied to tracking of a moving object in an up-down diagonal direction. In addition, the method of tracking the tracking point group between the detected object maps successive in time series and the method of determining the feature points described above can be used in combination, and the reliability of the tracking of the moving object can be obtained by combining the methods. Is improved. In addition, as the similarity determination, a method that is a normal method performed in image processing, such as a method in which the similarity is maximized when the sum of squares of the distances between corresponding constituent points or the sum of absolute values is minimum, or the like is used. Can be. In addition, although the description has been made using the rotation and parallel movement in two dimensions as the geometric transformation, the geometric transformation used in the present invention is not limited to this, and can be used for enlargement or reduction, or movement of an object in three-dimensional space. This includes spatial arrangement conversion of measurement points based on the above.
[Brief description of the drawings]
FIG. 1 is a perspective view conceptually illustrating a method for measuring an object position / shape using an ultrasonic array sensor according to an embodiment of the present invention.
FIG. 2 is a perspective view conceptually showing surface position / shape data obtained by the same measuring method.
FIG. 3A is a plan view showing an example of sensor arrangement in the measurement method described above, and FIG. 3B is a conceptual diagram of a detected object map obtained in the sensor arrangement in FIG.
FIG. 4 is a diagram of a detected object map obtained by the same measuring method.
FIG. 5A is a diagram of a detected object map obtained by the same measurement method, and FIG. 5B is a diagram of a reflection intensity map obtained by the same measurement method.
FIG. 6 is a flowchart of position / shape measurement according to an embodiment of the present invention.
FIG. 7 is a plan view illustrating occlusion in three-dimensional measurement.
FIG. 8 is a flowchart of a position / shape measurement and tracking method according to an embodiment of the present invention.
FIG. 9 is a view for explaining a tracking method according to an embodiment of the present invention.
FIG. 10 is a flowchart of a position / shape measurement and tracking method according to the embodiment.
FIG. 11 is a view for explaining another tracking method according to an embodiment of the present invention.
FIG. 12 is a flowchart of a position / shape measurement and tracking method according to the embodiment.
FIG. 13 is a flowchart of position / shape measurement according to an embodiment of the present invention.
FIG. 14 is a detected object map illustrating time-series data.
15A is a diagram for explaining geometric transformation of a matching pattern according to an embodiment of the present invention, FIG. 15B is a diagram for explaining occlusion correction, and FIG. FIG.
FIG. 16 is a flowchart of still another position / shape measurement and tracking method according to an embodiment of the present invention.
FIG. 17 is a diagram illustrating correction of a matching pattern according to an embodiment of the present invention.
FIG. 18 is a view for explaining a method for obtaining a feature point according to an embodiment of the present invention.
FIG. 19 is a view for explaining another method for obtaining the above feature points.
FIG. 20 is a flowchart of tracking point identification according to an embodiment of the present invention.
FIG. 21 is a view for explaining re-grouping of tracking points according to an embodiment of the present invention.
FIG. 22 is a flowchart of the same regrouping.
FIG. 23 is a view for explaining a method of tracking a tracking point group according to an embodiment of the present invention.
FIG. 24 is a view for explaining another method of tracking the tracking point group.
FIG. 25 is a block diagram of an apparatus for measuring the position and shape of an object according to an embodiment of the present invention.
[Explanation of symbols]
1,2 objects
100 ultrasonic sound source
200 Ultrasonic transmission control means
300 ultrasonic array sensor
400 Ultrasonic signal receiving means
500 Object position detecting means
600 Object position / shape measuring means
G, G (n), G1 (n), G2 (n) Tracking point group
L1 (n), L2 (n) Regression line
M (n) Matching pattern
OC occlusion
RM reflection intensity map
RU reflected ultrasound
S ultrasonic array sensor
T geometric transformation
V1, V2, V3 movement vector trajectory
ZM object map
ρ (n) radius of curvature

Claims (13)

Using the position detection data of the object detected by the ultrasonic array sensor, in the object position and shape measurement method of measuring the surface position and / or surface shape facing the sensor of the object that moves relatively to the sensor,
A map creation step of expanding the measured surface position and surface shape data into a detected object map,
Intensity map creating step of expanding the reflection intensity data at each point on the detected object map into a reflection intensity map,
A new detected object map is created by extracting, as reliable data, data that changes substantially smoothly within a predetermined range temporally or spatially for each of the data indicated in the detected object map and the reflection intensity map. And a data extraction step of performing the following. An occlusion estimating step of estimating an occlusion occurrence portion of the detected object by comparing the detected data of the surface position and / or the surface shape of the relatively moving object extracted two or more times in the data extracting step;
A pattern generation step of correcting a matching pattern used for tracking an object to the position and orientation of the object when the occlusion occurs, and generating a new matching pattern incorporating the occlusion,
2. The object position / shape measurement according to claim 1, further comprising: a tracking step of performing the next pattern matching using the generated new matching pattern to track the position and / or orientation of the object. Method. Obtaining the detected object map extracted in the data extraction step in time series, determining a predetermined feature point from a group of measurement points of the object for each detected object map as a representative point of the group, When the moving distance between the representative point of the previous group and the representative point in the subsequent group corresponding to this representative point is equal to or less than a predetermined amount, the former group and the latter group become 3. The method for measuring the position and shape of an object according to claim 1, further comprising a tracking step of determining the position and / or orientation of the object by determining that the object and the object correspond to the same group. Obtaining the detected object map extracted in the data extraction step in time series, determining a predetermined feature point from a group of measurement points of the object for each detected object map as a representative point of the group, If the difference between the reflection intensity of the representative point of the previous group and the reflection intensity of the representative point in the subsequent group corresponding to this representative point is equal to or less than a predetermined amount, the former group and the latter group 4. The apparatus according to claim 1, further comprising a tracking step of determining that the group corresponds to the same measurement point group of the object and tracking the position and / or orientation of the object. The method for measuring the position and shape of an object according to the above. The method according to any one of claims 1 to 4, wherein in the intensity map creating step, a measurement point having a reflection intensity equal to or more than a predetermined value is set as an effective measurement value in the detected object map. The detected object map extracted in the data extracting step is obtained in time series, and a group including measurement points of the object in the preceding detected object map in time series is used as a matching pattern as a geometrical pattern including movement and rotation. An occlusion detection step of performing conversion, superimposing this and a group in the subsequent detected object map to detect an object having a maximum similarity at a predetermined value or more, and detecting an occurrence portion of occlusion based on the detected object. ,
A tracking step of generating a corrected new matching pattern by incorporating the occlusion into the matching pattern, performing the next pattern matching using the generated matching pattern, and tracking the position and / or orientation of the object; 2. The method for measuring the position and shape of an object according to claim 1, further comprising: In the occlusion detection step, an occlusion occurrence portion is estimated based on shape data of an object known in advance, and the group subjected to the geometric transformation is corrected by the estimated occlusion occurrence portion to obtain a new matching pattern. 7. The method for measuring the position and shape of an object according to claim 6, wherein: 8. The method for measuring the position and shape of an object according to claim 6, wherein a portion having a large curvature of a curve formed by a continuous point sequence in the detected object map is included in the matching pattern as a feature point. When the curvature of the point sequence curve included in the group consisting of the measurement points of the object in the detected object map shows a change of not more than a predetermined amount in the plurality of detected object maps, the former group and the latter group are compared with each other. 9. The object position / shape measuring method according to claim 6, further comprising a tracking point identifying step of determining that the object points correspond to the same measurement point group of the object. For the first and second groups of measurement points of the object in the detected object map, the first group in the preceding detected object map in chronological order is subjected to a geometric transformation, and the first group in the later detected object map is subjected to geometric transformation. When a group corresponding to the first group is obtained, the same conversion is performed on the second group in the previous object map, and the points of the converted group and the points of the second group in the subsequent object map are used. 10. The method according to claim 6, further comprising the step of regrouping the first and second two groups into one group when the distance is equal to or less than a predetermined value. Method for measuring the position and shape of an object. 7. A trajectory prediction step for estimating a position of a group in a next detected object map from a trajectory obtained when a measurement point of a group of temporally continuous detected object maps is tracked. 10. The method for measuring the position and shape of an object according to any one of 10. When the slope of the regression line obtained from the point sequence in the group consisting of the measurement points of the object in the object map changes substantially smoothly in the plurality of object maps preceding and succeeding, the group of the preceding object map and the subsequent object map are changed. The object position / shape according to any one of claims 6 to 11, further comprising a straight line tracking step of judging that the detected object map group corresponds to the same measurement point group of the object. Measurement method. An ultrasonic sound source, an ultrasonic transmission control means for controlling the ultrasonic transmission of the ultrasonic sound source, an ultrasonic array sensor for detecting the ultrasonic waves transmitted from the ultrasonic source reflected by the detection object, and an ultrasonic array sensor. An ultrasonic signal receiving means for receiving an ultrasonic signal detected by the ultrasonic array sensor in synchronization with the ultrasonic wave transmission control means, and a processing object for processing each element output signal of the ultrasonic array sensor from the ultrasonic signal receiving means. Object position detecting means for detecting the position of the object, and surface position and surface shape data of the detected object are generated from the position detection data detected by the object position detecting means, and the generated surface position and surface shape data are developed on a detected object map. Map creation function means, intensity map creation function means for developing reflection intensity data at each point on the object map into a reflection intensity map, and the object map Data extraction function means for extracting data that changes substantially smoothly within a predetermined range temporally or spatially with respect to each data shown in the reflection intensity map as reliable data and creating a new detected object map An object position / shape measuring device, comprising: an object position / shape measuring means for measuring the position / shape of a detected object using the object.

Images