JP2993611B2 - Image processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial application field) The present invention captures a forward scene with two or more imaging cameras, and extracts or recognizes an object in the scene by stereo image processing. In particular, the present invention relates to extracting a planar portion such as a plane or a pseudo plane in the scene.

(Prior Art) For example, in a vehicle or a ship, a technology for automatically detecting a preceding vehicle, a ship, or an obstacle is desired. To respond to this, Japanese Patent Application Laid-Open Nos.
No. 44313 discloses a distance measuring device that performs stereo processing on image information of one screen each captured by two television cameras on a vehicle and calculates a distance of an object in a scene projected by the camera. Have been.

In the former, regions having the same brightness are divided into blocks on images taken by two cameras arranged at a predetermined distance from each other. Next, each block on one image is moved in a direction to reduce the parallax, and is compared with the other block to find a position where the blocks match. Calculate the distance from the amount moved until they match.

In the latter, regions having the same brightness are divided into blocks on images taken by two cameras arranged at a predetermined distance from each other. Next, the feature amount of each of the blocks on one image and the block on the other image is calculated, and the feature amounts are compared between the blocks captured by different cameras to detect the best matching block ( Detection of corresponding block). Then, the distance is calculated from the position difference between the corresponding blocks.

Further, Japanese Patent Application No. 2-29878 and Japanese Patent Application Laid-Open No. 29879/1990 filed by the present applicant disclose a photographic movie of a specific object in the scene, particularly an object having a complex outer surface with a large number of surface irregularities. A technique for detecting an area on a surface, that is, a texture area, and detecting a distance between each part of the surface of the object, that is, a three-dimensional shape is proposed.

(Problems to be Solved by the Invention) In the distance measuring device disclosed in the above-mentioned JP-A-61-44312 and JP-A-61-44313, when a parallax between two cameras is large, a very large area is formed. Since the correspondence search is performed, the probability of erroneously detecting the corresponding block increases when there are many blocks having similar characteristics. Further, since the search area is large, it takes time to detect the corresponding block.

The distance measuring device disclosed in the above-mentioned JP-A-2-29878 and JP-A-2-29879 has many advantages in detecting an object having a three-dimensional and relatively complicated shape. Is relatively monotonous and has a large area, and is difficult to appear on the screen as one independent body (single body), such as a table top surface, a floor surface, a road surface, a water surface, etc., a plane or a pseudo plane (hereinafter, referred to as a plane portion). Is difficult to extract. This is, of course, given that these distance measuring devices originally attempt to accurately extract a three-dimensional individual and more accurately detect surface irregularities.

By the way, for example, an automatic machine automatically processes, automatically cleans a table surface, a floor surface, a road surface, a water surface, or the like, or automatically moves an object on those surfaces, or runs by an operator. Automatic capture of other objects may not make sense. For example, while traveling on a curve, the front object captured by the television camera may be off the road, so that the capturing tends to be substantially meaningless. In addition, when there is parking on the road, or when traveling on a relatively wide road such as two or more lanes in one direction, even if there is an object (parking vehicle, vehicle traveling ahead, obstacle, etc.) ahead and it is not possible to go straight on, In order to determine whether it is possible to avoid such a situation, it is preferable to accurately detect a road surface area where no object exists (a road surface area where the object is excluded).

A first object of the present invention is to detect a planar portion region excluding a region where an object is present, and a second object is to easily and accurately detect the planar portion region.

[Configuration of the invention]

(Means for Solving the Problems) According to the present invention, a plurality of imaging cameras shoot scenes in front of them, obtain video signals, digitally process these video signals, and obtain the video signals with each imaging camera. In the image processing of searching for the corresponding point of the scene and extracting what is in the scene, detecting an object area located at the center in the horizontal direction and vertically below in each screen showing the scene Calculate the parallax of the corresponding point of the detected object area, detect the calculated substantially discontinuous change point in the two-dimensional direction as the edge of the plane or pseudo plane area, and touch the edge to change the parallax in the two-dimensional direction. Detects a substantially continuous area as a plane or a pseudo plane.

(Operation) Since the planar portion extends forward from below the camera, the planar portion is first widely reflected in the captured image on the screen and extends upward.

As shown in FIG. 7a, two stereo shooting cameras 2a and 2b at a certain height (h) from the plane shoot a scene diagonally downward and forward, and as shown in FIG. 1b, the left camera 2a Assuming that the photographing film surface of the right camera 2b is FLa and the photographing film surface of the right camera 2b is FLb, a point P (X, Y, Z) on the plane is a point (pixel: pixel) Pa on the left screen FLa, and a right screen FLb Appears at point Pb. Pa and Pb are the corresponding points on the left and right screens.

When only the front plane is reflected on both the left and right screens FLa and FLb, when the parallax is calculated for each corresponding point on the screen, the parallax continuously changes in the two-dimensional direction on the screen. small.

However, as shown in FIG. 7a, there is an object OB in front and the image is reflected on a part (for example, the center part) of the left and right screens FLa and FLb, and the front plane is reflected on the other part of the screen, Although the parallax is large on the surface OBfs of the object OB and substantially constant in the two-dimensional direction, the parallax shows a large change at a position outside the side of OBfs. In other words, if the left and right sides and the upper side of OBfs deviate outward, a distant plane is reflected there and the parallax is small, so a large sharp change appears in the two-dimensional parallax distribution. Although the rate of change of parallax is slightly small near the lower side of OBfs, the parallax continuously changes in two-dimensional directions because there is a plane between the camera and the object below the lower side, but the upper side of the lower side (plane OBfs) In this case, the parallax is substantially constant, and a considerable change also appears in the two-dimensional distribution of the parallax.

Therefore, a substantially discontinuous change point in the two-dimensional direction is detected as an edge of a plane or a pseudo-plane region, and a region in contact with the edge and in which the change of parallax in the two-dimensional direction is substantially continuous is defined as a plane or a pseudo-plane. Upon detection, the planar portion is extracted.

By the way, the planar portion appears on the photographic movie surface in such a manner that it extends right and left in the lower region within the screen and extends to the upper region. In particular, when an object that travels on a flat part, such as a vehicle, is equipped with a shooting camera, if the vehicle is more than a required distance from the object in front, the lower part of the screen in the cinematographic screen must be located in the screen. A flat area (road surface) appears. Therefore, on the screen, the probability that there is a planar portion in the lower area is the highest, and by detecting the object area located in the center in the horizontal direction and the lower side in the vertical direction in each screen, the left and right screens FLa, Detection of corresponding points of a scene on FLb, particularly detection of a planar portion and corresponding detection, are quick and accurate.

Other objects and features of the present invention will become apparent from the following description of embodiments with reference to the drawings.

(Embodiment) FIG. 1 shows a configuration of an image processing apparatus that implements the present invention in one embodiment.

This device consists of a computer 1, an imaging camera 2a (left), 2b
(Right), consists of an image memory 4, a display 5, a printer 6, a floppy disk 7, a keyboard terminal 8, and the like, which are mounted on a passenger car.

The left and right cameras 2a and 2b are ITV cameras (CCD cameras) of the same specifications, each of which captures a scene in front and outputs an analog image signal in a 512 × 512 pixel section. The D converters 22a and 22b convert digital data (image data) of 256 gradations per pixel. Movie camera 2a, 2b
Is a height h above a plane (road surface), as shown in FIG. 7a.
Diagonally downward with two stereos (downward angle)
To shoot the scene in front. As shown in FIG. 1b, the film FLa captured by the left camera 2a is a left image, and the film FL captured by the right camera 2b is F.
Lb.

The image memory 4 is readable and writable, and stores various processing data including the original image data of the left image and the right image.

The display unit 5 and the printer 6 output processing results of the computer 1 and the like, and
Registers the processing result. The keyboard terminal 8 is operated by an operator to input various instructions.

The computer 1 is further connected to a host computer, and controls each unit in accordance with an instruction given from the host computer or an instruction given from the keyboard terminal 8, and performs a corresponding process. Hereinafter, of the processing,
“Detection of road surface” for extracting a road surface region in a forward scene and displaying it will be described.

FIG. 2 is a main routine showing an outline of a "road surface detection" routine that is repeated in a substantially predetermined cycle in response to an instruction given from the host computer or the keyboard terminal 8 until the instruction is released. And 3a
FIG. 6 to FIG. 6 are subroutines showing the processing contents of the main processing items therein.

First, referring to FIG. 2, the computer 1 starts the left and right camera
The image data of the left and right images 2a and 2b is written into the image memory 4 (subroutine 1). In the following, the word "subroutine" or "step" is omitted in parentheses, and only the numerals attached thereto are shown. Next, image division of the left and right images is performed (2). Since the contents are the same as those disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 2-29878, the contents are not shown. The contents will be described.

In the "image division" (2), first, a forward raster scan is set for the left image. Note that the direction of the raster scan is the horizontal direction of the shooting scene (the left-right direction when looking forward from above the vehicle). In the forward raster scan, the ua parallel to Xa (Xb) in the imaging area FLa (FLb on the right screen) shown in FIG.
The (ub) axis is set as a main scanning axis, and the va (vb) axis parallel to Ya (Yb) is set as a sub-scanning axis, and scanning is performed on each pixel along a path from the upper left pixel to the lower right pixel. That is, the scanning in the u-direction (main scanning direction) and v (sub-scanning direction) shown in FIG. 7b is performed from the upper left pixel (image origin 0,0) to the lower right pixel (511,51
Perform until 1). Conversely, the backward raster scan described later is performed from the lower right pixel (511, 511) to the upper left pixel (0, 0). Differential data and gradient data are generated while performing a forward raster scan, and a histogram of the differential data is created.

The differential data is obtained by defining the original image data of the original image data of the target pixel and its eight neighboring pixels (3 × 3 image data matrix centered on the target pixel) as shown in FIG. 7D: (p1 + p2 + p8) It is the sum of the main scanning direction differential data given by − (p4 + p5 + p6) and the sub-scanning direction differential data given by (p2 + p3 + p4) − (p6 + p7 + p8), and indicates the spatial variation of the original image data. The inclination data is a ratio of the main scanning direction control data to the sub-scanning direction differential data, that is, the sub-scanning direction differential data / main scanning direction differential data is classified into eight categories. Indicates the direction in which it is changing. The computer 1 writes these data in the image memory 4 in association with each pixel.

FIG. 8a shows an example of a left screen image, and FIG. 8b shows a right screen image.

The histogram is obtained by calculating the size and the appearance frequency of the differential data. When the forward raster scan is completed, the computer 1 sets a threshold value from this histogram to binarize the differential data, It is stored in association with the pixel.

As a result of the binarization in this manner, a portion where the original image data changes abruptly spatially, that is, an “edge” is detected. When the edge indicates the outline or background of an object existing in the front scene, it is preferable that the edge be continuous. However, the edge is not necessarily continuous due to an imaging condition or the like. Therefore, a process of setting a forward raster scan again to connect discontinuous edges is performed.

That is, when an end point pixel (edge break pixel) of an edge which has not yet been subjected to connection processing is detected, the forward raster scan is interrupted, the pixel is set as a target pixel, and 3 × 3 pixels including eight neighboring pixels are set. Therefore, a pixel in the direction indicated by the inclination data of the target pixel is selected as an extension candidate pixel, and it is determined whether or not the pixel is an edge constituent pixel. Here, the extension candidate pixel and the differential data of the two pixels on both sides connected in a direction orthogonal to the inclination data of the pixel of interest are compared with each other, and if the differential data corresponding to the extension candidate pixel is the maximum, the ridge is determined.

If the ridge is determined, the edge is extended to the extension candidate pixel. At this time, if the extended pixel becomes the end point pixel again, the above is repeated by focusing on that. Otherwise, since the edge is connected, the forward raster scan is restarted. If it is determined that the pixel is not a ridge, the extension candidate pixel is updated to a pixel on the mountain side of the two pixels on both sides of the extension candidate pixel. At this time, if the updated extension candidate pixel is located in the eight pixels in the vicinity of the target pixel, the above-described determination is made. If the updated candidate pixel is out of that, the forward raster scan is restarted on the assumption that the edge has been cut there.

When the above image division is completed for the left image, image division of the right image is executed in exactly the same procedure.

The edge image shown in FIG. 9a (left image) and FIG. 9b (right image) is obtained by the “image division processing” as described above.

When the “image division” (2) is completed, the computer 1 next executes “extraction of road area” (3). The contents are shown in FIGS. 3a to 3d. This "extraction of road area" (3)
Calculates the size and center of gravity of each divided image (each area surrounded by edges), and extracts the area near the center where the size is above a certain threshold value and has the lowest center of gravity as the road surface area I do.

That is, referring to FIGS. 3a to 3d, when a forward divided raster scan is performed on an image to find a divided area (edge), the divided area makes one round along the edge. The sum of the number n (perimeter) of pixels on the edge of the region and the u coordinate of each pixel of the edge during one round is calculated, and the center of gravity (ug, vg) of the region is obtained by the following equation (1) (11 to 11). 16). In this calculation method, (ug, vg) is not an accurate position of the center of gravity, but a near-close position can be obtained at high speed. The number n of edge pixels is assumed to be the size of the region in a pseudo manner. The likelihood R as a road area is calculated from the obtained equation (2) (16). R is an evaluation function that has a larger value as the size of the region is larger, the center of the region is at the center, and the region is closer to the lower side of the screen. Therefore, R has a large value when the region in which it is calculated is the region indicated by oblique lines in FIG. 7b.

ug = (Σu) / n, vg = (Σv) / n (1) R = (uc-ABS (uc-ug)) × (2vc-Vg) × n (2) (uc, vc) : Camera center coordinates, ABS: Absolute value The above process is repeated for all the areas, the area having the largest R is found, and a label indicating that it is a road area is attached to each point in the area (17 to 22). At this time, if another area is included in this area, the included area is also temporarily labeled. Find edge pixels in the road area from the road area extraction results and the area division results, and label them as edge pixels in the road area (23 to 5
2). The same process is executed for the right image (53).

In FIG. 10a (left image) and FIG. 10b (right image), the road area and the labeled area are shown in black, and in FIG. 11a (left image) and FIG. 11b (right image), the road area and Only the edges of the region included in the region are shown.

Completion of "extraction of road area" (3)
Performs the process of "corresponding to left and right images" (4). The contents are shown in FIGS. 4a to 4c. In this case, the boundary of the region and the edge pixels of the region are associated with each other in the left and right images, the parallax of the corresponding pixel is calculated, and the three-dimensional position of each point of the edge is calculated based on the parallax and the pixel position. In binocular stereopsis, which takes correspondence with the road area edge pixels of the left and right images, how to eliminate erroneous correspondence is an important issue. As described above, there is a high probability that the road area exists below the screen in the vertical direction. Therefore, the left image is raster-scanned in the backward direction, and if a road area edge pixel (ua, va) is found, the corresponding point is predicted by the following methods and the corresponding point is determined.

Prediction from the relative position of the camera and the road When the positional relationship between the road surface and the camera is in the reference state (when there is no pitching of the vehicle), the imaging position of a point on the road surface by the left and right cameras is calculated. Predict the corresponding points of the left and right images. That is, a forward raster scan is performed, and the three-dimensional position (XaA, YaA, ZaA) in the left camera coordinate system of the point (on the road) corresponding to the edge pixel (ua, va) of the road area is calculated by the following equation (3). Calculate (54-57 in FIG. 4a). The three-dimensional position (XbA, YbA, ZbA) of this point in the left camera coordinate system is calculated by the following equation (4).
Then, the position (ubA, vbA) that should exist on the right image of this point (that is, the position on the right image based on the left image) is calculated (predicted) by the following equation (5) (57). Five pixels on the left and right (a total of 11 pixels in the main scanning direction) with the point as the center point are set as corresponding candidate points.

ZaA = Sya × h / (va−vca−Sya · tanθ) cosθ, XaA = (uca−ua) ZaA / Sxa, YaA = −ZaA · tanθ−h / cosθ (3) ZbA = ZaA, XbA = XaA −D, YbA = YaA (4) UbA = −XbA × Sxb / ZbA + Ucb, VbA = Va (5) Uca, Vca: Left camera center, Ucb, Vcb: Right camera center, Sxa, Sya: Left camera Scale factor, Sxb, Syb: Right camera scale factor, h: Camera height, θ: Camera downward angle, D: Distance between left and right cameras Prediction from continuous conditions The object represented by the divided image is three-dimensionally continuous If so, the image (region edge) of the target object continues on the two-dimensional image. Focusing on this, a point (pixel) in the vicinity of a point (pixel) already taken is predicted to be a corresponding point from their parallax. Therefore, this prediction is limited to the point where the parallax is obtained. If the parallax of the neighboring point has not been determined, the corresponding point is determined based on only the above prediction.

First, when predicting the corresponding point of the attention point (pixel) (ua, va), it is checked whether or not the disparity is registered from a point close to the attention point (ua, va) on the disparity map. That is, (u
a, va + 1), (ua, va + 1), (ua−1, va + 1), (ua
The disparity map is checked in the order of (+1, va + 1), (ua, va + 2). This order is shown in FIG. 7c. In FIG. 7c, the pixel marked with “1” is the point of interest (pixel). If there is a parallax, a predicted corresponding point (ubA, vbA) on the right image is calculated from the value (d1) by the following equation (6), and 5 pixels on the left and right (a total of 11 pixels in the main scanning direction) are used as corresponding candidate points. (58-67). here,
The disparity map is a memory area (memory table) in which the value of the disparity di of each point (pixel) (ui, vi) in the same uv coordinate system as the image is registered.

ubA = ua−d1, vba = va (6) Determination of correspondence The following equation is used between the point of interest (ua, va) on the left image and each of the correspondence candidate points (ub, vb) obtained in the above. The correlation value “I” is calculated according to (7), and the point where I is the smallest is determined as the corresponding point. The parallax d is calculated from the u coordinate of the corresponding point of the left and right images by the following equation (8) and registered in the parallax map (68 to 82). Then, from the parallax d, the three-dimensional position (Xa, Y
a, Za) is obtained by the following equation (9) (83).

I = ΣΣ (f (ua + j, va + i) −f (ub + j, vb + i) (7) i, j: −3 to +3 d = ua−ub (8) Za = Sxa × D / d, Xa = (Uca-ua) ZaA / Sxa, Ya = (xca-xa) ZaA / Sya (9) When an edge point of the road area is found while performing reverse raster scanning, the above-mentioned is executed to obtain parallax. As the process proceeds, the parallax map gradually fills up, and the probability of being able to make predictions gradually increases, and the parallax map is completed.

In this state, the road area includes an object area such as a preceding vehicle and an obstacle therein.

FIG. 11c shows a graphic display screen showing the three-dimensional position of the edge of the road area. FIG.
Based on the three-dimensional position information shown in FIG. 1c, a road area edge when viewed from above is displayed on a plane (planar projection), and FIG. 12b is a view when viewed from the front. FIG. 12c shows a road area edge as viewed from the right side in a plan view (right side projection), and FIG. 12c shows a road area edge as viewed from the right side. .

When the disparity map in the road area is completed as described above by “correspondence to left and right images” (4), the computer 1 executes “estimation of road surface” (5). Figures 5 to 5c
Shown in the figure. The vehicle travel path is usually the lowest point in the film scene. When viewed from a camera having a certain height with respect to the road surface, the lowest point is the farthest point (point with small parallax) on each horizontal line with the road surface.

Therefore, in “estimate road surface” (5), the lowest parallax of each horizontal line (main scanning line) on the parallax map is found, and this parallax is used as the parallax representing the distance to the road surface in that horizontal line. Register on the map. Next, the road surface is smoothly continuous, and the point of rapid change in parallax is different from that of the road surface (vehicles ahead, obstacles, curbs, guardrails,
A sharp change point is removed because the edge is a pole.

That is, referring to FIGS. 5a to 5c, the disparity map is scanned in the reverse direction by raster scanning to check the disparity data of the road area, and the past in the scanning order from the current point of interest (pixel).
From the parallax (dv-9 to dv) of 10 points (10 pixels), the following equation (10) is used.
It is checked whether the parallax (dv + 1) of the point of interest is a parallax representing the road surface (88 to 104). If the parallax is not a road surface parallax, dv + 1 'in equation (10) is updated and registered in the parallax map as the parallax of the point of interest (105). From the disparity (d) of each point of the disparity map calculated and corrected in this way, the three-dimensional position (Xa, Ya, Za) of each point in the left camera coordinate system is obtained by Expression (9) (106 to 118). ).

ABS (Dv + 1−dv + 1 ′) ≦ 2: road surface, ABS (Dv + 1−dv + 1 ′)> 2: road surface (10) dv + 1 ′ = a (v−1) + b, a = S1 / S2, b = ( Σd) / n−a (Σv) / n, S1: Σ (dv) − (ΣdΣv) / n, S2: Σv ² − (Σv) ² / n In this embodiment, n = 10, and (Ten)
Indicates that it is determined whether or not the deviation of the target point from the center value of 10 points obtained by the least square method is 2 or more.

FIG. 13a shows the area edge of the road surface (surface) based on the data of the disparity map obtained as described above, as viewed from above, and FIG. 13b, as viewed from the front, and FIG. 13c. Are shown as viewed from the right side.

After completing the "estimation of road surface" (5), the computer 1 executes "calculation of camera downward angle" (6). The contents are shown in FIG. In this case, the camera angle (downward angle) θ with respect to the road surface is calculated by the following equation (11) from the three-dimensional coordinates (X, Y, Z) of a point belonging to the boundary line of the road area by the least square method. The angle data stored in the angle register is updated to this. The initial value of the angle register is a standard state (ideal angle when the vehicle is in a standard state on a horizontal road surface), and is initialized (not shown immediately after the power is turned on to the system shown in FIG. 1a). Is written to the angle register.

θ = tan ⁻¹ (S3 / S4),... (11) S3 = Σ (ZY) − (ΣZΣY) / n, S4 = ΣZ ² − (ΣZ) ² / n, n = number of points on the boundary line When the processing of one cycle (1 to 6) is completed, the above processing is repeated in a form in which the processing of this one cycle is performed again after a predetermined time. Since the repetition is performed in this manner, following the change in the attitude of the vehicle, the “down camera angle calculation” (6) is executed to execute the down camera angle data θ.
Is automatically updated to the real one, and therefore the calculation error of the parallax, the distance, and the like due to the fluctuation of the downward angle θ due to the moment-to-moment posture change is reduced.

〔The invention's effect〕

In the present invention, for example, as an example in which a photographing camera is provided on a vehicle, first, only a road area (area of a plane portion) in a horizontal center portion and a vertical lower portion of a photographic film surface is first detected, and The parallax of each point is calculated by the left and right cameras, and the point where the parallax changes smoothly is extracted as a road surface (planar part), and the place where there is an object and the parallax changes rapidly is an object outside the road surface or on the road. Because only the road surface that is actually exposed while being excluded from the road surface is detected, the road surface detection processing time is short, and the road surface detection accuracy is high. When only the road surface area is detected on the vehicle as described above, automatic determination and automatic determination such as automatic determination of whether or not the vehicle may proceed in the current traveling direction and automatic deceleration or braking when it is determined that the vehicle is not traveling are performed. Control can be easily performed, and of course, the automatic turn control that changes the course to an empty road surface area and the automatic notification that prompts the driver to change to an empty road surface area based on the correlation between the traveling direction of the vehicle and the road surface area. Such control becomes easy.

[Brief description of the drawings]

FIG. 1a is a block diagram showing an apparatus configuration for implementing the present invention in one aspect. FIG. 1b is a perspective view showing the relationship between the images FLa and FLb captured by the left and right cameras 2a and 2b shown in FIG. 1a and a point P on a plane in front of the cameras. FIG. 2 is a flowchart showing an outline of a road surface detection processing operation of the computer 1 shown in FIG. 1a. FIGS. 3a, 3b, 3c and 3d are flowcharts showing the contents of "extraction of road area" (3) shown in FIG. FIGS. 4a, 4b, and 4c are flowcharts showing the contents of “correspondence to left and right images” (4) shown in FIG. FIGS. 5a, 5b and 5c are flowcharts showing the contents of "estimation of road surface" (5) shown in FIG. FIG. 6 shows the “camera downward angle calculation” shown in FIG.
It is a flowchart which shows the content of (6). FIG. 7a is a perspective view showing the relationship between the left and right cameras 2a and 2b shown in FIG. 1a, a point P on the road surface in front of them, and an object OB. FIG. 7b shows the forward raster scan origin (0,0), the center of the screen (uc, vc), and the horizontal center and vertically lower area (shaded area) in the screen of the left screen FLa shown in FIG. 1b. FIG. FIG. 7c is a plan view showing a search pixel of interest when searching for corresponding points on the left and right screens and reference pixels in the vicinity thereof in “correspondence between left and right images” (4) shown in FIG. 4a. . FIG. 7d shows the “image division” (2) shown in FIG.
FIG. 9 is a plan view showing a target pixel and peripheral pixels to be referred to in differential calculation for detecting an edge of an on-screen image, and symbols indicating the image density values. FIGS. 8a and 8b are plan views respectively showing screens projected by the left camera 2a and the right camera 2b shown in FIG. 9a and 9b are plan views each showing a screen obtained by performing the “image division” (2) shown in FIG. 2 on the screen shown in FIG. 8a and FIG. 8b. 10a and 10b are plan views respectively showing a screen obtained by performing “extraction of road area” (3) shown in FIG. 2 on the screen shown in FIGS. 9a and 9b. 11a and 11b are plan views showing images showing the edges of the road area and the edges of the area contained therein shown on the screen shown in FIGS. 10a and 10b. FIG. 11c is a plan view showing a graphic display showing the three-dimensional position of the road area obtained in “estimation of road surface” (5) shown in FIG. FIG. 12a is a plan view showing a plan view in which the three-dimensional position shown in FIG. 11c is converted into a two-dimensional position of horizontal plane projection. FIG. 12b is a plan view showing a front view in which the three-dimensional position shown in FIG. 11c is converted into a two-dimensional position of vertical front projection. FIG. 12c is a plan view showing a right side view in which the three-dimensional position shown in FIG. 11c is converted into a two-dimensional position of a vertical left side projection from the right when looking ahead from the vehicle. FIG. 13a is a plan view showing a road surface area by delineating the inside of the road surface edge of the planar projection shown in FIG. 12a. FIG. 13b is a plan view showing a road surface area by drawing the inside of the road surface edge of the front projection shown in FIG. 12b. FIG. 13c is a plan view showing the road surface area by drawing the inside of the road surface edge of the side projection shown in FIG. 12c. 1: Computer, 2a, 2b: Camera 22a, 22b: A / D converter 4: Image memory, 5: CRT display 6: Printer, 7: Floppy disk 8: Keyboard terminal

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-61-44312 (JP, A) JP-A-61-44313 (JP, A) JP-A-2-29878 (JP, A) JP-A-2- 29879 (JP, A) JP-A-2-73474 (JP, A) JP-A-4-127383 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G06T 7/00

Claims (1)

(57) [Claims] 1. A plurality of imaging cameras shoot scenes in front of them and obtain video signals respectively. These video signals are digitally processed, and corresponding points of the scene obtained by each imaging camera are searched. In the image processing of extracting an object in the scene, the corresponding point of the detected object area is detected by detecting the object area located at the center in the horizontal direction and located vertically below in each screen reflecting the scene. The parallax is calculated, and the calculated substantially discontinuous change point in the two-dimensional direction is detected as an edge of a plane or a quasi-planar area, and the area in contact with the edge and the change of the parallax in the two-dimensional direction is substantially continuous. An image processing method characterized by detecting a plane or a pseudo plane.