JP2002323301A - Object position measurement device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To stably detect an object even if distance information about a part of the object in lacking due to fluctuation of a lighting condition in stereo distance measurement. SOLUTION: This object position measurement device is characterized by being provided with an object detection processing selection means 4 judging whether the distance information is obtained from the overall figuration of the object to be detected or not, an overall figuration object detection processing means 5 deciding an objection position when the distance information is obtained from the overall figuration, and a partial figuration object detection processing means 6 identifying a missing part of the figuration by means of a previously prepared object figuration appearance model and computing a distance of the figuration about the missing part when the distance information about a part of the object figuration is missing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting an object reflected in an image based on distance information to each object in the image obtained by a stereo distance measuring method. Also, the present invention relates to a method for stably detecting an object and measuring a three-dimensional position of the object even when a part of the distance information of the object is lost due to a change in brightness of a part.

An apparatus for detecting an arbitrary object in an image and specifying its three-dimensional position includes, for example, a traffic flow measurement system that measures the position of a vehicle from a road image and calculates the presence or absence of a vehicle and the speed of passage. The present invention can be generally applied to an object monitoring system such as a person monitoring system that measures the position and the number of persons passing from an image of a person coming and going.

In a physical distribution system for determining the type of an article carried on a belt conveyor, the present invention can be applied to an article identification for specifying an article being carried based on the size and image characteristics of the article. Can be applied as an object detection means to identify obstacles on the traveling route with an unmanned transport system outside, and can be applied to recognition processing to measure the position and size of surrounding objects in an autonomous robot system It is.

[0004]

2. Description of the Related Art A conventional measuring method for measuring the position of an object using distance information around a sensor is based on a distance to each position of an object around the sensor by a stereo distance measuring method (this is called distance information). ) Is measured, and a part having a specific size or shape is extracted from the obtained distance information and identified as a corresponding object. Since the identified object has the distance information, the three-dimensional position can be measured from the stereo image viewed from the sensor based on the distance information.

As typical examples of this method, Keiji Sumiyoshi and Hanawa
Keiji, Nobuyuki Togawa, Kazuma Arai et al., "Forward Situation Recognition System for Driving Assistance Using Stereo Images" IEICE Technical Report PRMU 97-25-36pp.
The method of 39-46 will be described as an example.

According to this, as shown in FIG. 8, a scene including an object 21 to be measured is displayed on two television cameras 22 and 23.
To obtain a stereo image. Here, since a television camera mounted horizontally horizontally is used, a pair of left and right stereo images 24 and 25 can be obtained.

Next, from the stereo images 24 and 25, edge points, which are points at which the brightness suddenly changes, are extracted by differentiating the stereo image to obtain edge images 26 and 27 for the left and right images, respectively. Subsequently, by a stereo distance measurement process described later, a shift amount d is detected for each edge point as shown in an image position shift detection screen 28, and based on the shift amount d and the distance between the television cameras 22 and 23, The distance from the edge point to the object is measured.

[0008] The stereo distance measurement processing means that at each position in one edge image, for example, the edge image 26, N ×
A local region of N pixels is provided. Then, for each local area, a local area of N × N pixels is provided in the other edge image 27, and this local area is scanned in the other edge image 27, so that the same edge as the local area of the edge image 26 is obtained. A local area in the edge image 27 where points are distributed is detected. That is, the left edge image 26 and the right edge image 2
Then, a process of searching for a position indicating the same object from the position 7 is performed. This is the stereo distance measurement processing.

Since the position of the object, that is, the projected position is different depending on the distance in the left and right images, the projected position of the same object in the image is searched for from the left and right images. The distance to the object can be calculated.

The information of the distance obtained at each edge point of the image by the stereo processing is mapped to one point in the three-dimensional space. Get the distribution of locations. In this three-dimensional space, for example, a vehicle to be measured can be identified by detecting a three-dimensional lump that is above a predetermined road surface position and that matches the width, height, and depth of the vehicle. it can.

[0011]

In such a sequential method of acquiring distance information to each position of a target scene and measuring an object position based only on the distance information, the distance information to the object is obtained. The object can be identified,
The three-dimensional position can be measured.

However, when operating outdoors, for example, since the illuminance increases or decreases due to changes in sunlight or weather, the entire image becomes too dark or too bright, and the brightness of the target object and the background is reduced. May be similar, and the detection position of the edge point indicating the outer shape of the target object may be shifted.

[0013] In addition, since the positional deviation occurs in any of the left, right, upper and lower directions independently of each edge, the stereo point-by-edge stereo processing can be performed by comparing the edge point distribution shapes of the local regions. A matching part cannot be found from the left and right images, so that the distance of the object outer shape cannot be measured, or even if a matching part is found, the position of the edge includes a shift, so that incorrect distance information is presented. As a result, there has been a problem in that the distance information indicating the outer shape of the object is missing, so that the object cannot be detected, or that an erroneous recognition of an object at an erroneous position using erroneous distance information occurs.

In stereo processing using two images, a distance cannot be determined in principle for an edge in a direction parallel to the two cameras installed. For example, when cameras are installed horizontally horizontally, the distance of the horizontal line edge cannot be calculated, and the distance can be obtained only in the vertical outer shape of the object. When observing how the distance information of the object outer shape is missing when the illuminance change actually occurs, one of the outer shapes at the left and right ends of the object is often missing than when the entire object outer shape is not obtained. . If one of the left and right ends of the object outline is missing,
Since the three-dimensional width and height change, they cannot be detected as objects.

Considering such a situation, it is necessary that the position of the object can be accurately detected even when distance information cannot be obtained from a part of the outer shape of the object. Therefore, an object of the present invention is to provide an object position measuring device capable of accurately detecting the position of an object even when distance information cannot be obtained from a part of the outer shape of the object.

[0016]

FIG. 1 shows the principle configuration of the present invention.
Shown in In FIG. 1, 1 is a stereo image photographing unit, 2 is a stereo processing unit, 3 is an object detection processing unit, 4 is an object detection processing selection unit, 5 is an entire external object detection processing unit, and 6 is a partial external object detection processing unit. . The object of the present invention is as follows (1) to
This can be achieved by (3).

(1) In an object position measuring device for detecting an object from stereo measurement distance information, an object detection processing selecting means 4 for judging whether or not distance information has been obtained for the entire outer shape of the object to be detected; When the distance information is obtained for the entire outer shape, the entire outer shape object detection processing means 5 for determining the object position, and when the distance information for a part of the outer shape of the object is missing, the An object position measuring device, comprising: a partial external object detection processing means 6 for identifying a missing portion of an external shape using an external appearance appearance model and calculating a distance of the external shape to the defective portion. .

(2) With respect to the outer shape for which the distance cannot be determined by stereo processing for the missing portion, the outer shape is formed by the partial outer shape object detection processing means 6 based on a cluster of feature quantities such as edge points. The object position measuring device according to the above (1), wherein a stereoscopic correspondence is calculated for each block to calculate a distance of an outer shape of the missing portion.

(3) The overall external object detection processing means 5
And the partial external object detection processing means 6 calculates a plurality of object position candidates and sends them to the object position determination processing means, and the object position determination processing means uses information on the object positions obtained in the past. The object position measuring device according to (1) or (2), wherein a correct object position can be determined from a plurality of object position candidates.

As a result, the following operation and effect can be obtained.

(1) In addition to the processing unit for determining the object position when the distance is obtained for the entire object outline, even if the distance information of a part of the object outline is lacking, the object prepared in advance A processing unit is provided to calculate the distance even for a part where the outline is missing using a model of the appearance of the outline, so that not only when the distance is obtained for the entire object outline, but also for a part of the object outline The object can be accurately detected even when the object is missing.

(2) For a part of a missing contour whose distance is not obtained by the stereo processing, the contour is formed by using a lump having a feature amount different from the distance such as an edge point. Even for a chipped outer shape, an outer shape portion can be obtained accurately.

(3) Since the correct object position can be obtained in time series from a plurality of candidate object positions using the information of the object position obtained in the past, even if the background has an edge point, The object can be detected accurately without misidentifying the object.

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The outline of the present invention will be described with reference to FIGS. 1 is a schematic configuration diagram of the present invention, and FIG. 2 is an explanatory diagram of a processing state when a part of the outer shape is missing.

First, a stereo image obtained by a stereo image photographing section 1 having two cameras is sent to a stereo processing section 2 to acquire distance information. This distance information is obtained by the same method as in the conventional method. Differential processing is performed on the stereo image to detect an edge portion where the brightness changes abruptly, and then the same edge portion obtained from the left and right images matches. A part is detected, and a distance is obtained by calculating a deviation of the same part. Conventionally, when the object detection processing, and mapping the distance information in three-dimensional space, a model (three-dimensional model of the width, height and depth of the object from the 3-dimensional point information, what size detection object in advance , And only the process of detecting an object using a plurality of fixed objects or a variable size expressed by an equation) is performed by the entire outer shape object detection processing unit 5. I was

On the other hand, according to the present invention, in the process of detecting an object from the distance information, not only the case where the distance is obtained for the entire object outline but also some distance information of the object outline is missing. In this case, a model of the appearance of the external shape of the object prepared in advance is used to identify where the external shape is missing. The processing is performed by the processing unit 6, and the processing when the distance of the entire outer shape is obtained and the processing when the distance information of a part of the outer shape of the object is lacking are controlled by the object detection processing selecting unit 4 so as to be switched. The object can be detected stably even when the entire object outline is obtained or when a part of the object outline is missing.

The processing when a part of the outer shape of the object is missing is performed by the partial outer shape object detection processing unit 6 in FIG. 1 and is an important component of the present invention. The process of calculating the distance of the outer shape again at the portion where the error has occurred will be described with reference to FIG. In FIG. 2, it is assumed that the camera is mounted horizontally in the left-right direction and the shape of the object is rectangular (rectangular parallelepiped), and only the left outer shape of the object can obtain distance information.

First, the appearance model M of the outer shape of the object is applied to a distance image (left distance image) in which the distance is determined for each edge position of one image (here, the left image captured by the left camera). Check whether the external shape is missing (processing). This detects that the right end of the outer shape is missing.

Next, for the portion determined to be missing (the right end in this case), the edge image is directly examined, and the presence or absence of an edge point for which the distance corresponding to the outer shape of the object has not been determined (process).

If there are edge points for which the distance has not been determined, those edge point groups are extracted as one lump (process). Then, this edge block is regarded as the right end of the missing outline (process).

This operation is similarly performed on the other image (here, the right distance image), and an edge block forming the right end of the outer shape is detected (processes ',', ',').

Next, the distance information outline obtained from the left distance image is set as the outline left end (processing), and the outline right end obtained by the processing is integrated to create a left integrated screen. Similarly, the outer contour of the distance information obtained from the right distance image is defined as the outer left end (process '). The right outer edge obtained from the above process' is integrated to create a right integrated screen. Then, the right masses of the left integrated image and the right integrated image are associated with each other (for example, the average position is calculated), and the distance of the right outer shape is calculated (process).

As described above, as in the case where the difference in brightness between the object and the background is reduced, the edge point itself exists, but the accuracy of the detected position is low and the correspondence between the edge points between the left and right images, that is, Even if it is impossible to detect the coincidence of the edge points, it means that the clumps of the edges are associated with each other.

However, if the object outline is determined only by using the edge points, there may be a case where the object outline is erroneously recognized by using an edge of an erroneous portion which is not an object such as a background edge. Therefore, in the present invention, as will be described in a second embodiment described later, the partial external object detection processing unit 6 determines the model matching degree (evaluation value of the likeness of an object), and searches for the object position while searching for the object position. A plurality of top K-th (K is 1 or more) object candidates having a high model matching degree are output. The object position is determined by determining the smoothness of movement based on past object position information such as the history information of the object's motion, and the most likely object among the plurality of candidates is finally determined. Is output as a simple object position. Thus, even if a part of the outer shape of the object is missing, the correct object position can be determined without being affected by the background and other objects.

One embodiment of the present invention will be described with reference to FIGS. 3 is an embodiment of the present invention, FIG. 4 is an explanatory diagram of a distance image, FIG. 5 is an explanatory diagram of a scanning range,
FIG. 6 is a diagram illustrating the relationship between the appearance model of the object outline and the outline detection area.

In the figures, the same symbols as those in the other figures denote the same parts, 1 is a stereo image photographing section, 2 is a stereo processing section, 3 is an object detection processing section, 4 is an object detection processing selection section, and 5 is an entire external object. The detection processing unit 6 is a partial external object detection processing unit.

The stereo image photographing section 1 is composed of two or more, for example, two cameras having different installation positions, and obtains, for example, two stereo images by this stereo camera device.

The stereo processing unit 2 receives the stereo image obtained by the stereo image photographing unit 1 as input, first performs a classification process on each of the input images, and extracts a portion where the brightness changes suddenly, that is, an edge point. Create an edge image. Next, a stereo association process for obtaining the same portion is performed by using an edge image of the stereo image as an input, and a distance is obtained for the same portion from a shift amount of the image. Then, a distance image having depth information is created for each edge point only for the object for which the distance is required, and is output from the stereo processing unit 2. At the same time, an edge image group obtained first is also output as an output of the stereo processing unit 2 for subsequent processing.

The object detection processing unit 3 receives the distance image and the edge image group which are the outputs of the stereo processing unit 2 as inputs,
It performs a process of detecting an object to determine a three-dimensional position, and includes an object detection processing selection unit 4, an entire external object detection processing unit 5, a partial external object detection processing unit 6, and the like.

The object detection processing selection unit 4 determines the input distance
From the information (distance image), the part P that looks like an object_i(_i≧ 1)
(Called an object fragment) and extract the tertiary
Object shape model M representing original shape_j(_j≧ 1)
Model ML in the imageⁱ _j(_j≧ 1) is the object fragment
P_iThe object outline is missing in the distance information
It is to judge whether or not. The appearance model is an object
The body is shown according to the position where it is seen, and the same object
However, the shape changes depending on the visible position.
For example, in the case of a rectangular parallelepiped, the view model is rectangular when viewed from the side
, And when viewed from the diagonal side, the appearance model is the front part
In addition to the shape, the side is visible. In the example of Fig. 2, the cuboid is true.
The appearance model M when viewed from the side is shown.

If there is no chipping, the object detection processing selection unit 4 shifts the processing to the whole outline object detection processing unit 5 and judges the presence or absence of the object using the distance information of the whole outline of the object. Determine the position. If the object detection processing selection unit 4 determines that there is a lack in the object outline, it shifts the processing to the partial outline object detection processing unit 6 and uses the distance information of the partial outline and the edge point information in the part where the outline is missing. Is determined, and if there is an object, its position is determined.

The details of each part will be described below.

First, the object detection processing selection unit 4 will be described.
I do. In the object detection processing selection unit 4, first, the object is detected in the following procedure.
Body fragment P_i(0 ≦_i≤ N; N is the number of object fragments)
You. P_iFirst, as shown in FIG.
The width and height of the distance image Dimg are G_w× G_hSmall area G_k
(Grid), edge points included in each small area
The average and variance values of the distance distribution and the number of edge points
Area or edge where the variance of the distance is greater than the threshold
In the area where the number of points is below the threshold, the distance is not constant and the area is unstable.
Delete as area. For the remaining small areas, the distance image
Small area G in upper (two-dimensional)_kDistance difference L between₁And small area
Has a distance difference (depth difference) L in a three-dimensional space _TwoBut it
Small area groups within each threshold are combined into one object fragment
P_iExtract as It is located three-dimensionally close
This means that the object is extracted as an object fragment.

Next, for each of the obtained object fragments P _i ,
Using the three-dimensional shape model M _{j of the} object (1 ≦ j ≦ Nm; Nm is the number of types of the prepared object shape models), the object outline when each of the models is placed at the position (distance) of the object fragment P _i in There how "object outline appearance model ML ⁱ _j" indicating how looks (i is the number of object fragments, number of model j are using, 1 ≦ j ≦ Nm; object shape model Nm is to prepare Number of types). 3-dimensional shape model of the object may be of any type but, as one where an implementation, the width MW _j of the object, the height MH _j, depth MD _j
Is adopted.

When the front of the rectangular parallelepiped model is placed so as to match the center position (Cx _i, Cy _i, Cz _i ) of the three-dimensional position of each object fragment P _i , each vertex of the rectangular parallelepiped becomes (Cx _i ± MW _j /
_{2, Cy i ± MH j /} 2, Cz i) × (Cx i ± MW j
/ 2, Cy _i ± MH _j / 2, C _{z i} + MD _j ) (the sign combination is arbitrary). With each vertex position being (X, Y, Z), the projection position (x, y) on the image is calculated using the focal length f, which is an internal parameter of the camera, and the distance b between the two cameras.
y) is obtained by the following equation (1), and the minimum rectangular area surrounding the projection point

[0046]

(Equation 1)

The appearance model ML ⁱ _j (1 ≦ 1)
j ≦ Nm; Nm is the number of types of prepared object shape models). Continue scanning the inside distance image in the size of the appearance model ML ⁱ _j of the object outline, if satisfying the judgment condition (1) described below, considered as a whole object contour is Motoma'. At this time, a k (i) -th object candidate O ^ij _{k (i)} is generated, its scanning position is recorded for O ^ij _{k (i)} , and a flag for shifting to the whole external object detection processing unit 5 is set. Stand up. This k (i) is a serial number assigned to the object candidate detected for the i-th object fragment P _i , and a different number is assigned to each candidate, meaning that a plurality of candidates may be detected. I do.

When the judgment condition (2) described later is satisfied, it is regarded that a part of the object outer shape has been obtained, and k (i)
The second object candidate O ^ij _{k (i)} is generated, its scanning position is recorded, and a flag for shifting to the partial external object detection processing unit 6 is set.

If neither condition (1) nor condition (2) is satisfied,
The object is assumed to be absent and scanning continues. Scanning range, as shown in FIG. 5, the range from the most left end object fragment P _i and object appearance model ML ⁱ _j overlaps up to the rightmost, i.e., the model appearance and the right of the object fragments P _i ML
^{The range} is from the position where the right side of ⁱ _j coincides to the position where the left side of the object fragment P _{i and} the left side of the appearance model ML ⁱ _j coincide.

Judgment condition (1): (inspection of whether or not the entire outer shape has been obtained) Here, when the direction in which the stereo cameras are arranged is considered to be a horizontal axis (it is not necessary to actually be horizontal), Cameras (images) located on the left and right
To the right. At this time, due to the nature of the stereo, only the vertical component of the object can determine the distance. Therefore, what is important as the object outline is the vertical outline at the left and right ends. Therefore, the presence or absence of the outer shape at the left and right ends is used as a criterion for determining whether or not the entire outer shape has been obtained.

[0051] established a appearance model ML ⁱ _j of the object contour in the distance image, the inspection region, as shown in FIG. 6 (R1~R
How to see the 3) to set the position of the model ML ⁱ _j to the reference.
Here, R1 is a region for examining the existence of the left end of the outline, R2 is a region for examining the existence of the outline on the right end, and R3 is a region for determining whether the object is inside or outside. The size of the area is determined as follows. R1, for R2, the height the same as the height MH ⁱ _j of the appearance model ML ⁱ _j, width R _w
It shall be given in the function F (MW ⁱ _j) in the reference width MW ⁱ _j of appearance model ML ⁱ _j, for example F (X) = X
/ 4. As shown in FIG. 6, the positions where R1 and R2 are set are such that 1/2 of the width of R1 or R2 overlaps the appearance model. R3 is R in the appearance model.
1, a part other than R2. Therefore, the width of the region R3 is M
In _{^{W i j -F (MW i j}} ), a height of MH ⁱ _j.

When both of the following conditions are satisfied for the detection regions R1 and R2 of the outer shape, it is determined that all the outer shapes of the object have been found.

(The number of edge points having a distance within the inspection area R1 on the left end) ≧ threshold Th ₁ (the number of edge points having a distance within the inspection area R2 on the right end) ≧ threshold Th ₁ in, the threshold Th ₁ is a threshold for suppressing the influence of noise that defines the minimum value of the number of edge points having a distance to be in a region R1 (or R2). When there are more edge points than this, it is considered that there is a cluster of edge points having a stable distance, that is, there is an outer shape having a distance.

Judgment condition (2): (inspection of whether a part of the outline is obtained, that is, whether part is missing) Similar to the judgment condition (1), the appearance model ML of the object outline is determined.
established a ⁱ _j in one of the distance image of the stereo image, the investigation region (R1, R2) shown in FIG. 6, it is checked whether satisfies one of the following conditions.

[0055]

(Equation 2)

Here, the "edge point for which the distance has not been determined" means that the position of the edge point is shifted due to a small change in brightness between the object and the background, and the left and right edge images are subjected to stereo processing. It indicates an edge point for which the same location cannot be determined and the distance is not determined. If either of these conditions is satisfied, at one end there is an object outline having distance information that can determine a three-dimensional position, and at the other end, a feature (trace) that can form the object outline as a lump although the distance is unknown. ) Is present. Trout, distance has not been determined threshold Th ₂ for the number of edge points, against a threshold Th ₁

[0057]

(Equation 3)

The values have the following relationship. Where T in the equation
Is an average correspondence ratio with respect to all edge points, which is obtained in advance by, for example, an experiment, and represents a ratio of the number of edge points whose distance can be determined on average to the total number of edges. Normally, the distance is not obtained for all the edge points, so the value is 1 or less. By using this function T, adjustment is made so that the number of edge points for which the distance is obtained and the number of edge points for which the distance is not obtained are substantially equal.

If any of the conditions of Equation 2 is satisfied, it is determined that there is a possibility that a part of the outer shape of the object has been found (partly missing), and the following processing will be described. On the other hand, the presence / absence of the outer shape is similarly examined in the image. The purpose of this study is to combine parts of different objects,
This is to prevent erroneous recognition that the object is a part of which lacks an outer shape, and confirms whether it is possible to determine that "the outer shape is missing" without inconsistency between stereo images.

If neither equation 2 is satisfied,
It returns that the judgment condition (2) is not satisfied.

The determination of the presence or absence of the outer shape of the object in the other image, which is the next process, is performed as follows. First, based on the average distance on the “region including many edge points having a distance” in the outer shape detection region R1 or R2 in the original image, the position where the object should be seen in the other image is calculated (described later). Next, the investigation area shown in FIG. 6 is set at a position to be seen in the other image, and the following conditions are set in the area on the same side as the “area including many edge points for which the distance is not determined” in the original image. Find out if you are satisfied.

(The number of edge points for which the distance is determined) ≦ threshold Th ₃ (the number of edge points for which the distance is not determined) ≦ threshold Th ₂ This condition is satisfied between the stereo images on the same side of the left and right ends of the object. This is for judging that a contour having a distance is missing. If the above two conditions are satisfied, since a part of the outer shape is missing without contradiction, it is returned that the determination condition (2) is satisfied. If they are not satisfied, it is determined that the judgment condition (2) is not satisfied.

On the other hand, the calculation of the position where the object should be seen in the image is performed as follows. On the other hand, the edge point position E _{1 of the} image
(X ^e _1, y ^e ₁ ) and the edge point position E ₂ (x
The relationship of ^e _2, y ^e ₂ ) is related as follows by the distance D _{E of the} edge point, the focal length f of the camera, and the distance b between the cameras (in the case of a horizontally installed stereo).

[0064]

(Equation 4)

[0065] Thus, by using the average distance Dmean the side where been determined distance of the object outline, by shifting the rectangular region of appearance model ML ⁱ _j of the object outline in the x-direction by the calculated value content, on the other hand the image Of the object can be determined.

If scanning has been completed for all the object models for the i-th object fragment P _i , the above operation is repeated for the other object fragments P _k (k = _{i + 1} ). If the above processing has been performed on all the object fragments, the processing of the object detection processing selection unit 4 is completed, and the object detection processing (the entire outer shape object detection processing ₎ associated with each of the object candidates ^Oij _{k (i)} is completed. The processing is shifted to the section 5 or the partial external object detection processing section 6).

Next, the entire external object detection processing section 5 will be described.
explain. In the whole external object detection processing unit 5, the object fragment P
_iThe overall shape with the distance obtained by scanning with respect to
Object candidate O obtained from^ij _kAbout the existence of the object
Is determined, and if it is determined that there is an object, the three-dimensional
Determine the position. Details of the processing will be described. First,
For the i-th object fragment, the j-th object model (look
Model MLⁱ _j) Scan k)
Supplement O^ij _k(0 ≦ k ≦ NO^j _i; NO^j _iIs the i-th object
Obtained when the j-th model is scanned for fragments
Number of object candidates), the appearance model MLⁱ _jThis
To evaluate the degree of fit when fitting
Model agreement E1^ij _kAnd calculate the object fragment P_iFor each
Model among the candidate objects obtained from all the appearance models
Value E1 at which the degree of coincidence becomes maximumⁱ _{ma x}And give it a mo
The Dell position (j, k) is obtained as follows.

[0068]

(Equation 5)

If the maximum value E1 ⁱ _max is equal to or larger than the threshold value,
The position (object appearance model ML ⁱ ) given by (j, k)
_It is determined that the object exists at the position where _j is set at the position where the k-th object candidate is given). If the maximum value E1 ⁱ _max is less than or equal to the threshold value, it is determined that no object exists for the object fragment P _i .

Here, the degree of coincidence E1 ^ij _k with the appearance model
Is a quantity representing how much the feature quantity of an object obtained from a given model and an image matches, but there is no particular mention of a method of realizing it. In the contour detection areas R1 and R2 shown in FIG. 6, the total number of edge points having a distance is E1 ⁱⁿ _1, and the contour detection area R among the edge points having a distance in the object inside detection area R3.
1, the score whose difference from the average distance of R2 is within the threshold is E
Assuming that 1 ⁱⁿ ₂ and the number of points equal to or larger than the threshold value are E1 ^out , a model matching degree is given by the following equation.

E1 ^ij _k = E1 ⁱⁿ ₁ + E1 ⁱⁿ ₂ −E1 ^out This is because the edge point having the same distance as the object from the outer shape or the inside of the object can be obtained, and the object under inspection from the inside of the object. The higher the number of edge points having different distances, which is evidence other than that, the higher the score is, and the object is considered to be an object.

Next, the partial outline object detection processing section 6 will be described. In this processing, when the outer shape having the distance obtained by scanning with respect to the object fragment P _i can be obtained only from a part (either the left or right end), the presence or absence of the object is determined.
If it is determined that there is an object, the process is to determine its three-dimensional position. The details of the process are the same as those of the whole external object detection processing unit 5 except that the method of calculating the degree of coincidence with the appearance model is different.

The appearance model coincidence E2 ^ij _k in the object detection processing in the partial outline is the same as that in the object detection processing in the entire outline, and is based on the characteristic amount of the object obtained from the given model and image. The method of realizing this is not particularly described as long as it is an amount indicating whether or not they match. For example, the following method is used.

E2 ⁱⁿ _{1 is} the number of edge points having a distance on the side including the edge point having a distance in one of the outer shape detection regions R1 and R2 shown in FIG. 6, and the other distance cannot be obtained. An evaluation value obtained from the number of edge points for which the distance is not obtained in the outer shape detection area on the side including the edge point is E2 ⁱⁿ _2, and the outer shape detection area among the edge points having the distance in the object inner detection area R3. When the difference between the average distance obtained from the side having the distance in either of R1 and R2 is within a threshold value, E2 ⁱⁿ ₃ , and the score, which is equal to or larger than the threshold value, is E2 ^out , the model matching degree is given by the following equation.

[0075] ^{_{E2 ij k = E2 in 1 +}} E2 in 2 + E2 in 3 -E2 out , however, E2 ⁱⁿ ₂ = (the number of edge points of the outline detection area)
× T (T is the same average correspondence rate as in Expression 2) The meaning of this expression is the same as that of the whole external object detection processing unit 5 described above, and an edge point having the same distance from the external portion or the inside of the object as the object is obtained. The higher the score is, the higher the score is that the edge points having different distances from the inside of the object, which are evidence of the outside of the object under inspection, are not obtained. However, in order to make the influence of the number of edge points for which the distance is obtained equal to the number of edge points for which the distance is not obtained, E2 ⁱⁿ
It is multiplied by the count T in the _second calculation.

A second embodiment of the present invention will be described with reference to FIG. 7, the same symbols as those in the other figures denote the same parts, 10 is an object detection processing unit, 11 is an entire external object detection processing unit, 12 is a partial external object detection processing unit,
Reference numeral 13 denotes an object position determination processing unit.

The object detection processing unit 10 receives distance information and edge information from the stereo processing unit 2 and outputs an object position, similarly to the object detection processing unit 3 shown in FIG. The difference is that the whole external object detection processing unit 11 and the partial external object detection processing unit 12 constituting the object detection processing unit 10 are different from the whole external object detection processing unit 5 in FIG.
And each object fragment P
_A plurality of K (K ≧ 2) object position candidates U ⁱ _k (k represents the number of object position candidates and 0 ≦ k ≦ K) are selected instead of determining only one object position for ^i. To do
The object position determination unit 13 is newly provided with the plurality of object position candidate U ⁱ _k, using the information of the past object, from a plurality of object position candidate U ⁱ _k, each object fragment P _i
Is determined.

With this configuration, it is possible to determine the position of the past object that best matches the information, such as, for example, movement information, without determining the object position using only the information obtained from the image. The object position can be correctly determined.

Hereinafter, portions different from the configuration of FIG. 3 will be described in detail.

First, the overall external object detection processing section 11 will be described. FIG 3 shows that different compared to the overall outer shape object detection processing section 5, i-th j th appearance model ML ⁱ k-th object obtained by scanning a _j with respect to the object fragment P _i for the candidate O ^ij _k, appearance model using ML ⁱ _j a model matching degree E1 ^ij _k to evaluate good or bad degree fitting when fitted, for each object fragment P _i, obtained through all appearance model used Object candidate ^Oij _k (0 ≦
From i ≦ N, 1 ≦ j ≦ Nm), the object position candidates U ⁱ _k (0 ≦
k <K).

Next, the partial outline object detection processing section 12 will be described. FIG 3 shows partial outline object detection processing unit 6 and to different points comparison, evaluation as before, the object candidate O ^ij _k, fitting good degree bad when fitting the appearance model ML ⁱ _j The object candidates O ^ij _k (0 ≦ i ≦ N, 1 ≦ j ≦ N) obtained through all the appearance models used for each object fragment P _i using the model matching degree E2 ^ij _k
m), the object position candidates U ⁱ _k (0 ≦ k ≦ K) are output from the object having the highest model matching degree to the top k-th.

Next, the object position determination processing section 13 will be described. Now, it is assumed that N ₀ objects W _q ( _q represents the number of the object and 0 ≦ q ≦ N ₀ ) have already been detected.
It has a kinetic model to each of each object W _q, from the history of the object position up to the previous, to predict the object position W ^pre _q which may be present time using a motion model. The actual motion model is not particularly defined, and may be, for example, a linear prediction model from a position one hour before or a prediction model using a Kalman filter. For a certain predicted object position W ^pre _q , an object position candidate U ⁱ _k obtained for each of the N object fragments P _i
(0 ≦ k ≦ K), and if the three-dimensional distance difference is within the threshold, the predicted object position W ^pre _q and the object position candidate U ⁱ _k
Object position from ^{_{W pre q = F p (W}} pre q, U i k) as to update the object position W ^pre _q. Then, the remaining object position candidates U ⁱ _k that do not correspond to W ^pre _q cannot exist at the same time, and are deleted from the object position candidates.

By this operation, the object position W^pre _qAnd corresponding
Object position candidate U for which was not foundⁱ _k(0 ≦ k ≦ K, 0
≦ i ≦ N), each object fragment P_iObjects detected from
Body position candidate Uⁱ _kThe highest evaluation value for each (0 ≦ k ≦ K)
If a new object is detected at a new candidate position,
W^pre _{q + 1}Register as As a result, the detected object
Number N₀← N₀+1.

Further, object position candidates U ⁱ _k (0 ≦ k ≦ K,
If there is an object position W ^pre _q that does not correspond to any of 0 ≦ i ≦ N), the object position W ^pre _q is deleted. As a result, the number of detected objects is set to N ₀ ← N ₀ -1.

In this way, it is possible to extract the one closest to the predicted position, so that the edge of the part other than the object such as the background edge is not mistaken for the object outline, and the past outline is not used. The most likely object can be accurately output as the final object position using the information on the object position, that is, the smoothness of the movement based on the movement history information and the like.

The present invention relates to a method for detecting an object from distance information, not only when distance information is obtained for the entire object outline but also when part of the object outline lacks distance information. Uses a model of the appearance of the outline of the object prepared in advance, identifies where the outline is missing, adds processing to calculate the distance of the outline again at the missing part, and switches between these two methods as appropriate Therefore, even if a part of the object outline is missing, the object can be detected stably, so that the brightness of the entire image increases or decreases due to changes in the lighting conditions, etc., and as a result, distance information of a part of the object is missing In this case, the object can be detected stably.

[0087]

According to the present invention, the following effects can be obtained.

(1) In addition to the processing unit for determining the object position when the distance is obtained for the entire object outline, even if the distance information of a part of the object outline is lacking, the object prepared in advance A processing unit is provided to calculate the distance even for a part where the outline is missing using a model of the appearance of the outline, so that not only when the distance is obtained for the entire object outline, but also for a part of the object outline The object can be accurately detected even when the object is missing.

(2) For a part of the missing outline for which a distance cannot be obtained by the stereo processing, the outline is formed by using a lump having a feature amount different from the distance, such as an edge point. Even for a chipped outer shape, an outer shape portion can be obtained accurately.

(3) Since the correct object position can be obtained in time series from a plurality of candidate object positions using the information of the object position obtained in the past, even if the background has an edge point, The object can be detected accurately without misidentifying the object.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of the present invention.

FIG. 2 is an explanatory diagram of a process when a part of an outer shape is missing in the present invention.

FIG. 3 is an embodiment of the present invention.

FIG. 4 is an explanatory diagram of a distance image.

FIG. 5 is an explanatory diagram of a scanning range.

FIG. 6 is a diagram illustrating the relationship between a visual appearance model of an object outline and an outline detection area.

FIG. 7 is a second embodiment of the present invention.

FIG. 8 is an explanatory diagram of position shift detection using a stereo image.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 stereo image photographing unit 2 stereo processing unit 3, 10 object detection processing unit 4 object detection processing selection unit 5, 11 overall external object detection processing unit 6, 12 partial external object detection processing unit

 ──────────────────────────────────────────────────続 き Continuing on the front page F term (reference) 2F065 AA04 AA12 AA51 BB05 BB15 BB24 CC11 CC16 DD04 FF05 JJ03 JJ05 JJ26 KK01 MM02 QQ06 QQ13 QQ24 QQ25 QQ27 QQ34 QQ39 QQ42 RR08 SS02 UU05 FA06 CA06 BA06 CA06 AA16 AA19 BA11 CA13 CB06 CB13 CB20 CE12 DA11 DA12 DA13 DB03 DC16 DC30 5L096 AA09 BA08 CA05 DA02 EA07 FA06 FA70 HA01 HA07 HA09 JA24

Claims (3)

[Claims] 1. An object position measuring apparatus for detecting an object from stereo measurement distance information, wherein distance information excluding a horizontal component and a stereo camera whose distance is not obtained by stereo measurement in the contour of the object to be detected is obtained. Object detection processing selecting means for determining whether or not the object has been obtained, and when distance information has been obtained for the entire outer shape, an entire outer shape object detection processing means for determining an object position; and a part of the outer shape of the object. If distance information is missing,
A part-outside object detection processing means for identifying a part where the outside is missing using a previously prepared appearance appearance model of the object, and calculating a distance of the outside to the missing part. Object position measurement device. 2. An outline for which a distance cannot be determined by stereo processing for the missing part is formed by the partial outline object detection processing means based on an aggregate of feature quantities such as edge points. 2. The object position measuring device according to claim 1, wherein the distance of the outer shape of the missing portion is calculated by performing stereo correspondence in units. 3. The whole external object detection processing means and the partial external object detection processing means calculate a plurality of object position candidates and send them to an object position determination processing means.
3. The object position measuring device according to claim 1, wherein the object position determination processing means is capable of determining a correct object position from a plurality of candidate object positions using information on the object position obtained in the past. .