JPH0229879A - Image processing method - Google Patents

Abstract

PURPOSE:To properly divide a texture area by binarizing respective image data in fine sections constituting an original image, dividing the spacially high density area of the binary data and dividing the texture on the basis of superposition to the spacially high density area of the binarized data. CONSTITUTION:Respective image data of picture elements in fine sections constituting an original image are binarized to form a binarized data, the spacially high density area of the binarized data is divided and the texture area is divided on the basis of the area divided by the spacial continuity of the binarized data and superpotion to the spacially high density area of the binary data. Since a certain kind of constitutional element exists in the texture area at comparatively high density, the density can be easily divided. Since the area division is also based upon superposition to the areas divided by the spacial continuity of the binary data in addition to the density, the boundary of the areas can be cleared and proper division can be attained.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial Application Field) The present invention relates to image segmentation of an image in which regions having various properties coexist.

(Prior Art) A typical image includes regions having various properties. Texture plays an important role in visually determining the identity of each area6.
For example, in an image containing a melon, the texture (pattern) of the melon separates the area of the melon from other areas, and the distant texture looks fine and the nearby texture looks coarse, so the perspective , that is, it expresses a three-dimensional shape. In other words, when performing image processing, if a texture region can be divided rationally, it is convenient for various types of processing related to the object indicated by the region, such as stereoscopic display using distance detection and recognition processing.

Generally, in the texture area, attempts are made to extract the constituent elements based on the fact that some basic constituent elements are arranged under certain rules and to analyze the arrangement rules.

(Problem to be Solved by the Invention) By the way, if this is followed, in the case of a melon, the feature amount will be detected for each line element that makes up the mesh, and it will be statistically processed. The amount of processing is extremely large and the amount of data to be processed is enormous, so it is expected that a microcomputer with a small memory capacity will be able to handle the Great Seal.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing method that appropriately divides texture regions from an image containing regions having various properties through simple processing.

[Structure of the invention]

(Means for Solving the Problems) In order to achieve the above object, the present invention binarizes each image data of pixels of minute sections constituting the original image.
Generate digitized data, divide a region with high spatial density of the binarized data, and divide the binarized data into regions based on its spatial continuity and the spatial density of the binarized data. Divide the texture region based on its weight with the high region.

shall be taken as a thing.

(Operation) According to this, since a certain type of component exists in a texture region at a relatively high density, the present invention, which focuses on the density, can easily divide this. At this time, the boundary of the area becomes clear because it is based not only on the density but also on the overlap with the divided area based on the spatial continuity of the binarized data.

Appropriate division is done.

In a preferred embodiment of the present invention, the binarized data is obtained by binarizing change data indicating spatial changes in image data. Thereby, the constituent elements of the texture can be detected more clearly than in the case of simple binarization, and their continuity can be ensured.

Other objects and features of the present invention will become apparent from the following description of embodiments with reference to the drawings.

(Embodiment) FIG. 1a shows the configuration of an image processing device implementing the present invention by way of example.

This device consists of a computer 1. ITV camera unit 2
and 39 image memory 4. Display 5. It consists of a printer 6, a floppy disk 7, a keyboard terminal 8, etc. ...ITV camera units 2 and 3 each have an ITV camera 21° 31 with the same specifications that images a scene in front and outputs analog data in 512 x 512 pixel divisions, and output analog data from the ITV camera 21 or 31. It consists of an A/D converter 22.32 which performs digital conversion to generate 256-gradation original image data.

The ITV cameras 21 and 31 are shown in FIG.
(However, this shows a model of imaging by ITV cameras 21 and 31, and does not show the structure of each camera.) They are arranged horizontally with IT'V camera 21 on the left and ITV camera 31 on the right. , the respective optical axes 211 and 311 are parallel, and the respective imaging planes 2
12 and 312 are on the same plane. Therefore, the upper side of each imaging plane is the XL axis, the X axis is the left side, and the left side is
When the YL axis and the YR axis are used, the XL axis and the X axis are on the same straight line, and the YL axis and the YR axis are parallel to each other. In the following, the image captured by the ITV camera 21 will be referred to as a left image, and the image captured by the ITV camera 31 will be referred to as a right image.

The image memory 4 is readable and writable, and stores various processed data including original image data of the left image and right image.

The display unit 5 and printer 6 output the processing results of the computer 1, the floppy disk 7 registers the processing results, etc., and the keyboard terminal 8 is operated by the operator to input various instructions. Ru.

A host computer is further connected to the computer 1, and controls each section according to instructions given from the host computer or instructions given from the keyboard terminal 8, and performs corresponding processing.

Among these processes, "stereoscopic processing" for displaying three-dimensional graphics of the scene in front will be described below.

FIG. 2 is a general flow showing an outline of a stereoscopic processing routine that is activated by a stereoscopic processing execution instruction given from the keyboard terminal 8, and FIGS. 3a to 3h are subflows showing important steps therein in detail. It is.

Computer 1 starts stereoscopic processing.

Sl (indicates the number assigned to the step in the flowchart:
After initializing the image memory 4 and the registers used in this process in S2 (hereinafter the same meaning) and writing the original image data of the left image and right image into the image memory 4 via the ITV camera units 2 and 3, in S3 , performs image segmentation of each image.

This image division will be explained with reference to FIG. 3a.

In 5101, forward raster scan is set for the left image. Forward raster scanning is performed within the imaging area 212 (312) shown in Figure 1b, with the XL (XR) axis as the main scanning axis and the YL (YR) axis as the sub-scanning axis, from the upper left pixel to the lower right pixel. This is a scan that focuses on each pixel along the path, and here the scan start position is initialized. Incidentally, the reverse raster scan, which will be described later, refers to a scan in which each pixel is focused on a path from the lower right pixel to the upper left pixel.

In steps 8102 to 5105, differential data and slope data are generated without performing forward raster scanning, and a histogram of the differential data is created.

The differential data is (PI +P2 +P8) - (P4 +Ps +P6) when the original image data of the pixel of interest and its 8 neighboring pixels are called as shown in Figure 4.
) and the main scanning direction differential data given by (P2+P
3+P4) This is the sum of the sub-scanning direction differential data given by (P6+P7+P8), and indicates the amount of spatial change in the original image data. In addition, the tilt data is the ratio of the above-mentioned main scanning direction differential data and sub-scanning direction differential data, that is, sub-scanning direction differential data/main scanning direction differential data is divided into 8 categories, and the original image data is spatially It shows the direction of change. The computer 1 writes these data into the image memory 4 in association with each pixel.

The histogram determines the size and appearance frequency of the differential data. When the computer 1 completes the forward raster scan, the computer 1 sets a threshold value from this histogram in step 5106 to binarize the differential data, and converts the binarized data into individual data. It is stored in association with the pixel.

As a result of binarization in this manner, areas where the original image data changes rapidly spatially, ie, "edges" are detected. When these edges indicate the outline or texture of an object existing in the front scene, it is preferable that the edges be continuous, but depending on imaging conditions and the like, they are not necessarily continuous. Therefore.

When the forward raster scan is set again in step 5107, processing for connecting discontinuous edges is performed in steps 8108 and below.

In step 8108, when an end point pixel of an edge (edge break pixel) that has not yet been subjected to connection processing is detected, the forward raster scan is interrupted and that pixel is set as the pixel of interest, and in step 5109, the pixel of interest is A pixel in the direction indicated by the tilt data is selected as an extension candidate pixel, and the determination is made in 5110. Here, the extension candidate pixel and the differential data of two pixels on both sides connected to it in a direction orthogonal to the tilt data of the pixel of interest are compared,
If the differential data corresponding to the extension candidate pixel is maximum, it is determined to be a ridge.

If it is determined that the edge is a ridge, the edge is extended to that extension candidate pixel in step 5111. At this time, if the extended pixel becomes an endpoint pixel again, return to 5112 to 5109 and repeat the above, noting it, but if not, the edge is connected, so proceed to 5115 and perform forward raster scan. resume. If it is determined that the pixel is not a ridge, the extension candidate pixel is updated in step 5113 to the pixel on the mountain side of the two pixels on both sides of the extension candidate pixel.

At this time, if the updated extension candidate pixel is in the 8th neighborhood of the pixel of interest, the process returns to 5110 and performs the above determination; however, if it is outside the range, the edge is assumed to be cut and the process proceeds to 5115 to restart forward raster scanning. do.

After completing the above image segmentation for the left image, image segmentation for the right image is performed using exactly the same procedure.

Figures 5a and 5b show the results of image segmentation for a left image and a right image, respectively. From this, the contours of the pot, melon, cup, and saucer and the texture of the melon are clearly expressed in line drawings. I can see that I am being ignored.

Once the computer 1 has completed the image segmentation, it extracts the high edge density regions of each image at 84, which will be described with reference to FIG. 3b.

Here, a forward raster scan is set for the left image in step 8201, and a loop consisting of steps S202 to S5213 is executed.

5202, nXn pixels (
In this example, n=21) is set, register Me is cleared in 5203, and 820
4, a forward raster scan within the window is set.

8205 to 8208 are loops that count the edge constituent pixels within the window, and the number is stored in the register Me.

8209 to 3211 are all pixels within the window (n
Edge density is determined from the ratio (Me/nXn) of edge constituent pixels (value of register M6) to If so, "high" (a label indicating a pixel in a high edge density region) is written corresponding to the pixel of interest.

When the above extraction of the high edge density region for the left image is completed, extraction of the high edge density region for the right image is executed in exactly the same procedure.

Figures 6a and 6b show the above-mentioned 5aI! 1 and 5b are the results of extraction of high edge density regions, respectively, and the extracted regions are indicated by black pixels. These figures include the 5th al! From Figures I and 5b, an area indicating a melon, which is naturally expected to be a high edge density area, has been extracted, but its outline is not very clear.

Once the computer 1 has finished extracting the high edge density regions, it labels the regions of each image at 85, which will be described with reference to FIG. 3c.

In step 5301, forward raster scanning is set for the left image, and in step 5302, the crawling data is set to an initial value L for the ``--like surface region.''

If the pixel of interest is an edge constituent pixel and has not been labeled, the forward raster scan is interrupted, registers Mt and Mr are cleared in 5305, and the edge is cleared in a loop consisting of 5306 to 5311. Without tracking, label data L is written in association with the pixels forming the edge, the number of pixels forming the edge is counted in the register Mr, and the register MtII,
The number of pixels included in the high edge density region of that edge is counted. In this edge tracking, an edge in a certain direction with respect to the tracking direction is selected every time there is a branch.

Therefore, as shown in FIG. 7a, when edges constitute a plurality of closed regions, the periphery of the smallest unit of the closed regions (for example, R1) is tracked. In addition, the count value of the register Mr indicates the number of peripheral pixels of the closed region, and the count value of the register Mt indicates the number of peripheral pixels of the closed region included when images (indicated by hatching) extracted from the high edge density region are superimposed. Each number represents a number.

At 5312, the ratio between the count value of register Mr and the count value of register Mt is determined. This ratio indicates the proportion of traced edges that are included in the high edge density region, and when this ratio is less than a predetermined value M s ``'' (0.5 in this example), --like edges are detected. After determining that it is a surface area and incrementing the label data L by 1 in S314, forward raster scanning is restarted, but when it exceeds a predetermined value Ms", it is determined to be a texture area, and the edge is tracked again in steps 5315 to 5317. Therefore, after the label data written in association with the pixels constituting the edge is rewritten to t'', the forward raster scan is restarted.

Through the above processing, the edges of the --like surface area are given labels that identify them, and the edges of the texture area are given a common label t''. This means that the edges of the texture area are As shown in Figures 5a and 5b, there are an infinite number of edges, and it was judged that it would be impractical to label them independently.6 For example, in the case of Figure 7a, the edges constituting each small area are Since all or most of the pixels are included in the high edge density region, the label data corresponding to all the pixels is t''.

In the loop consisting of 5321 to 5327, forward raster scanning is not performed, and the same label as the edge is attached to the area closed by the edge.6 In other words, when the label data of the pixel of interest remains initialized ( 0), reads the label data attached to images that are continuous in the upper, lower, left, and right directions in each direction, and if there is label data that is common to each direction, that label data is written as the label data of the pixel of interest, and the common label data is written as label data of the pixel of interest. If there is no label data, label data ``'A'' indicating that it is a background pixel is written as the label data of the pixel of interest. However, in this case, there is an area closed by another edge within the area closed by an edge, and When there is a plurality of label data common to the pixel, the label data of the pixel of interest is selected from among the plurality of common label data that appears in the latest order in the raster scan.

For example, when there is an area A2 surrounded by edges with label data L2 within an area Al surrounded by edges with label data L1, in raster scanning, pixels outside area A1, pixels within area A1 and areas Pixels outside A2,
Focusing on pixels in area A2, pixels in area A1 but outside area A2, and pixels outside area A1, in that order, and when focusing on pixels outside area A1, there is no common label data, so label data "Δ '' is written as label data of the pixel of interest, and when focusing on a pixel within area A1 but outside area A2, common label data L1 is detected, so it is written as label data of the pixel of interest, and
When paying attention to the pixels within, the common label data L1
and L2 are detected, and the label data L2 that appears later is taken as the label data of the pixel of interest.

Processing similar to the above processing for the left image is executed for the right image.

FIGS. 8a and 8b show the results of performing the above processing using the data constituting the aforementioned FIGS. 5a and 6a or FIGS. 5b and 6b, and showing only the texture area. These figures and figures 6a and 6b
Comparing with the figure, it can be seen that the area indicating the melon is clearly extracted. This is shown in Figure 7b,
This is because the boundary is continuous because the high edge density region was extracted based on the edge.

When the computer 1 finishes labeling the area, it returns 86
In this step, the texture regions of each image are integrated and labeled. This will be explained with reference to FIG. 3d.

In step 5401, forward raster scan is set for the left image, and in step 5402, label data is set to an initial value T'' for the texture area.

In the texture area, the common value t" is written as label data by the above-mentioned labeling, so 540
3 and 5404, a pixel in the texture area is searched without looking at it. If the pixel is a region boundary pixel and has not been processed, forward raster scanning is interrupted and 8
In the loop consisting of 407 to 5409, the region boundary pixels are tracked, and label data is assigned to each pixel by T.
Rewrite it to .

Finishing this for one texture area leaves 54
Since the label data T is incremented by one in step 10, when the forward raster scan is completed, a label for identifying each texture area is attached corresponding to the boundary pixel of each texture area.

Next, in a loop consisting of steps 5414 to 5419, a forward raster scan is performed, and the labels attached corresponding to the boundary pixels of each texture area are attached to all pixels in that area. Every time a pixel in each texture area that can be distinguished by label data t1 is detected, the pixel to the left of it (hereinafter left pixel = peripheral pixel or processed pixel) is detected.
This is done by reading the label data corresponding to the target pixel and using it to rewrite the label data corresponding to the pixel of interest.

Processing similar to the above is also performed on the right image.

In this way, as a result of executing the processes from 83 to S6, the --like surface area and texture area are divided into units that are recognized as independent areas for each left and right image.

Subsequently, in S7, the computer 1 associates the boundaries of each area of the left image with the boundaries of each area of the right image using feature amounts. This will be explained with reference to FIG. 3e.

In 5501, the following 5502 to 5 for both left images
The forward raster scan process of the loop consisting of 512 is set.

At 5502, the label data corresponding to the pixel of interest is read, and if it is not "A", the pixel of interest is -
Since it belongs to a similar surface area or texture area, 55
04, the pixel counter associated with the label data is incremented by 1, and the original image data of the pixel of interest (
Then, the square of the original image data of the pixel of interest is added to the register that stores the sum of squares gradation associated with the label data.

If the pixel of interest at this time is an unprocessed boundary pixel, the forward raster scan is interrupted, and in a loop consisting of steps 5507 to 5509, the boundary pixel is traced while looking at the area to the left, and the pixel of interest and its four Obtain the vector data of the pixel of interest by linear approximation from the coordinates of the pixel at the toe (in tracking), and write it in association with the pixel of interest. When this is completed,
In 8510, the vector data obtained at that time is referred to, the boundary line of the area is segmented based on the proximity of the vector data and the continuity of pixels, and the label of each segment, the coordinates of the start point and end point, and the label of the area to which it belongs are placed in a table. Summarize.

After this, restart the forward raster scan and repeat the above to complete the segmentation of the boundaries of each region,
Since the number of constituent pixels and the sum and square sum of the original image data corresponding to the constituent pixels are determined for each region, in step 5513, these data are used to find the average gradation and gradation variance of each region.

The number of constituent pixels, average gradation, and variance of each region are the feature amounts of each region.

In step 5514, segmentation of the boundaries of each area in the right image and feature amount detection of each area are performed in the same manner as above.

In step 5515, each region of the left image is associated with each region of the right image. At this time, the feature amount of the area of the left image,
The condition is that the feature amounts of the regions of the right image that are the candidates are equal within a permissible range.

In step 8516, segments of the boundaries of mutually correlated regions in the left and right images are correlated.

Here, the start and end points of both segments are on the same epipolar line within a permissible range, the slope and length of the line segment connecting them are equal within a permissible range, and the direction in which the area exists with respect to the line segment is equal. The condition is that. In this way, by associating the boundary line segments of the left and right images, the parallax between the pixels constituting the boundary line segments of the left image and the pixels constituting the boundary line segments of the right image (by overlapping each image deviation in the main scanning direction)
is determined and stored in association with the former (pixels forming the segment of the boundary line of the left image).

In step S8, the computer 1 correlates the boundaries of each texture area in the left image with the boundaries of each texture area in the right image.

This will be explained with reference to FIG. 3f.

In 5601, the following 5602 to 5 for the left image
The forward raster scan process of the loop consisting of 618 is set.

In 8602 to S6Q5, when a boundary pixel of an unprocessed texture area is detected, the forward raster scan is interrupted and the process proceeds to 5606 and subsequent steps.

Since the unprocessed boundary pixel has parallax data with the corresponding pixel in the boundary line association using the feature amount described above, it is read in 3606 and stored in the register D.

5607, the total sum (
mXm'Xm harmonic) is calculated and stored in registers E and eo, and in 8608, parallax data D
The value obtained by subtracting 2 from the value of register D (same symbol is used for convenience: same meaning for others) is stored in register i, and the value obtained by adding 2 to it is stored in register I, respectively.

mXm'Xm of the pixel of the right image whose parallax with the pixel of interest is i in 5609 (hereinafter referred to as the parallax 1 pixel of the right image)
Harmony is calculated and stored in the register ei, and in 5610, the difference between the value of the register eo and the value of the register ei, that is, m
The m'Xm harmonic difference and the value of register E are compared. At this time, if the former is less than the latter, the former is stored in register E in 5611, and the value of register i is stored in register D.

Thereafter, register i is incremented by 1 at 5612, and the above process is repeated until the value exceeds the value of register I.

In other words, in this embodiment, pixels that are highly correlated with each other in the left and right images are on the same epipolar line.

And assuming that the surrounding gradations are approximately equal, 5609
~5613, the right pixel (the right pixel with the highest correlation) that has the m It is selected from pixels.

In 5614, the value of register E, that is, the difference in each m x m ' sum gradation between the pixel of interest and the right pixel selected at this time, is used as correlation data, and the value of register D, that is,
The parallax of the right pixel is stored as parallax data in association with the pixel of interest.

At step 5615, the boundary pixels are updated by looking at the area to the left, and the processing from step 8606 is repeated.

When the correlation based on the above-mentioned correlation is completed for the boundary line of one texture area, forward raster scanning is restarted, and the boundary lines of all the texture areas are associated.

FIG. 9a shows the boundary line of the texture area of the left image, and FIG. 9b shows the right image as a result of correlation-based correspondence.
That is, it shows the right pixel indicated by the parallax data of the pixels forming the boundary line of the left image.

After completing the correspondence between the boundaries of the texture areas of the left and right images as described above, next, in S9, forward raster scanning is performed, and based on the parallax data of the left boundary of each texture area, The pixels inside the pixel are associated with each other.

First, the concept of the process will be explained with reference to FIGS. 10a, 10b, and 10c.

For example, while performing a forward raster scan of the left image and moving the pixel of interest po (symbols shown in Figure 4 apply mutatis mutandis; the same applies to the others), the constituent pixels of the left boundary line BL of a certain texture area Assume that the pixel to the right of is detected. In this case, the pixel to the left of the pixel of interest po (left pixel)
ps is the constituent pixel of the left border line BL, and the pixel immediately above (upper pixel) P3 is the pixel that has completed processing, each of which has parallax data Q or U. In this embodiment, adjacent pixels are Assuming that they have substantially equal parallax data, these parallax data Q and U are applied to the pixel of interest po to obtain the parallax 2 pixel Po' of the right image, the 4 pixels before and after it along the main scanning axis, and the right image. The pixel po'' on the parallax and the four pixels before and after it along the main scanning axis are determined, and the correlation with the pixel of interest Pa is examined.Furthermore, inside the pixel po'', the same examination is repeated using the processed parallax data. .

This will be explained in more detail with reference to FIG. 3g.

In 5701, forward raster scan processing is set for the left image, and in 5702 to 5705, unprocessed pixels in the texture area are searched for. When this is detected, 87
Proceed to 06 and below.

At 8706, the parallax data of the pixel to the left of the pixel of interest is read and stored in register Q, and the parallax data of the upper pixel is read and stored in register U, respectively.

In 5707, the m x m ' sum gradation of the pixel of interest is calculated and stored in registers E and 6Q, and in 8708, the value obtained by subtracting 2 from parallax data U is stored in register i.

The loop consisting of 8710 to 5714 is the above-mentioned 5609
The same processing as the loop consisting of ~5613 is performed, and the right pixel with the mXm'Xm harmonic closest to the mXm'Xm harmonic of the pixel of interest (the right pixel with the highest correlation) is selected as the right pixel specified by the parallax Q and the main scanning Select from among the four pixels before and after it along the axis, and calculate each mXm'Xm of the pixel of interest and the pixel to its right.
The harmonic difference is stored in register E, and the parallax of the right pixel is stored in register D.
are stored in each.

Next, in a loop consisting of 5717 to 5721, exactly the same processing is performed using the parallax data U. This allows the pixel with parallax 0 in the right image and the 4 pixels before and after it, and the pixel with parallax U in the right image and the 4 pixels before and after it. , the right pixel with the highest correlation is selected.

Register E stores the correlation data, and register D stores the parallax data for that pixel.
At step 22, the data is stored in association with the pixel of interest.

As explained above, this process can be said to propagate the parallax data of the left boundary line to each pixel inside the left boundary line. Therefore, as the pixel of interest moves away from the left border, the uncertainty of its disparity data increases.
At 510, the correspondence is now corrected by a reverse rask scan.

For example, see Figures 11a-11c.

Perform reverse rask scan of the left image to find the pixel of interest Pa.
Assume that a pixel to the left of a constituent pixel of the right boundary line B of a certain texture area is detected while moving the texture area.

This pixel of interest Po has parallax data D obtained in the above-mentioned initial correspondence processing.

Also, the pixel to the right (right pixel) ps is the right border line 8
Since these pixels are constituent pixels of the 6th generation, naturally these parallax data should be approximately equal if they had parallax data r, but the pixel of interest Po at this time is far from the left border line BL. As mentioned above, it may be different when you are far away. In that case, the parallax pixel P of the right image.

The correlation with the pixel of interest Pa is examined for one parallax pixel Po'' of the right image and four pixels before and after it along the main scanning axis.Furthermore, inside it.

A similar study will be performed using processed parallax data.

This will be explained in more detail with reference to FIG. 3h.

In 5801, reverse rask scan processing is set for the left image, and in 5802 to saos, unprocessed pixels in the texture area are searched for. When this is detected, 8
Proceed to 806 and below. At '5806, the correlation data of the pixel of interest is read and stored in register E.
The parallax data is read and stored in register D, the parallax data of the pixel to the right of the pixel of interest is read and stored in register r, and in 5807, m x m of the pixel of interest is stored.
'Calculate the Japanese gradation and store it in register g(,.

At 8808, the disparity data D of the pixel of interest and the disparity data r of the right pixel are compared, and the difference between them is set to a predetermined value Pg'
If it is less than (2 in this embodiment), the correlation data E and parallax data D of the pixel of interest are determined, but if not, the processing from 8809 onwards is performed.

The steps below 8809 are the same as above, and the parallax pixels of the right image, the 1 parallax pixel, and 4 pixels before and after it are calculated using the parallax data r.
The right pixel with the highest correlation with the pixel of interest is selected from among the pixels, and the correlation data and parallax data stored corresponding to the pixel of interest are each corrected.

After completing the processing up to 810, the computer 1
calculates the distance from the installation position of each ITV camera unit to the point on the object corresponding to each pixel using parallax data associated with each pixel. In this regard, please refer again to FIG. 1b.

As a result of imaging the point Q on the same object with the ITV cameras 21 and 31 and performing the above processing on the image,
Assume that it is found that the pixel QL of the left image corresponds to the pixel QR of the right image.

In other words, point Q is a straight line connecting the focal point oL of the ITV camera 21 and the pixel QL of the imaging surface 211, and the point Q
Focus of 0. and the straight line connecting the pixel QR of the imaging surface 311.

Therefore, if the distance 2a between the optical axes of the ITV cameras 21 and 31 and their focal length f are used, the pixels QL and QR
From the parallax with Z = 2 a f / D ・=
- As (1), the distance Z to the point Q on the object is found.

In Sll, distance data to the corresponding point on the object is obtained for each pixel according to equation (1), and in S12, the object is displayed three-dimensionally on the display 5 using the distance data. Figure 12 shows an example.
The processing result of the same original image as shown in FIG. Referring to this diagram:
The spherical curved surface of the melon is well expressed, and it can be seen that the correspondence result between the left image and the right image according to this example is correct.

To summarize the above, in this embodiment, the ITV camera 2
The following processing is applied to the left image and right image captured in steps 1 and 31.

(1) Perform image segmentation by differentiating each image and extracting edges; (2) Extract regions with high spatial density of edges for each image; (3) Extract areas with low edge density in each image Each region is given an individual label, and regions with high edge density are given a common label. (4) Among the regions with high edge density in each image, a group of regions that can be recognized as independent regions Label each region individually as a texture region; (5) Segment the border of each extraction region in each image, and associate the segments of the left image and the segments of the right image based on the feature amount; (6) ) Correlate the boundary line of the texture area in the left image with the boundary line of the texture area in the right image based on correlation; (7) Use the disparity data of the left boundary line of the texture area in the left image to find the pixels in that area. Obtain parallax data; (8) Correct the parallax data of the pixels in the area using the parallax data of the right border of the texture region in the left image; (9) Calculate the parallax data of each pixel from the camera installation position using the parallax data of each pixel. Find the distance to the point on the object.

Performs 3D display.

In other words, in this embodiment, since the image is divided into texture areas and non-texture areas, appropriate processing can be performed on each area. In this division, attention is paid to the fact that elements forming the basis of the texture area are arranged with a certain degree of density within the texture area, so the processing is simple and appropriate.

In addition, when determining the correspondence between the left image and the right image, note that the parallax between adjacent pixels is approximately equal, first correlate the boundaries of the regions, and then use the parallax of the left border to After determining the parallax within, this is corrected based on the parallax of the right border. This can be done easily and accurately even in a case where there are many constituent elements such as a texture area, and it is difficult to make correspondences using feature quantities.

Note that in the above embodiment, two images, left and right, are used, but if more images are to be processed, the same processing as described above may be performed for each.

It should also be noted that techniques such as edge extraction, labeling of pixels within a region surrounded by edges (intra-region propagation of edge labels), and boundary segmentation are merely examples.

〔Effect of the invention〕

As described above, according to the present invention, attention is paid to the fact that a certain type of component is present in a relatively high density in a texture area, so this texture area can be easily divided. At this time, since it is based not only on the density but also on the overlap with the area to be divided based on the spatial continuity of the binarized data, the boundaries of the area become clear and appropriate division is performed.

Furthermore, as shown in the example, if binarized data is obtained by binarizing change data indicating spatial changes in image data, the constituent elements of texture can be obtained more easily than by simply binarizing. This makes it possible to clearly detect and ensure continuity, making the division of texture regions more reliable.

[Brief explanation of the drawing]

FIG. 1a is a block diagram showing the configuration of an image processing device implementing the present invention as an example, and FIG. 1b is a schematic diagram showing a model of imaging by the ITV cameras 21 and 31 shown in FIG. 1a. Figures 2, 3a, 3b, 3c, and 3d. Figures 3e, 3f, 3g and 3h are 1a
3 is a flowchart showing the operation of the computer 1 shown in the figure. FIG. 4 is a plan view showing the spatial relationship between the pixel of interest and eight neighboring pixels. Figures 5a, 5b, 6a, 6b, 7a,
Figure 7b, Figure 8a, Figure 8b, Figure 9a, Figure 9b,
10a, 10b, 10c, 11a, 11b, 1ie, and 12 are plan views showing specific examples of the processing of the computer 1 shown in FIG. 1a. 1: Computer 2.3: ITV camera unit 21.31: I TV camera 22.32: A/D converter 4: Image memory 5: Display 6: Printer 7: Floppy disk 8: Keyboard terminal Figure 1b Figure 1a L Figure 58 Figure a? IK Sb Figure ri IJ Mouth η Sa r': <+ Figure 1I Figure Xi [XNth (pregnancy)) Figure 0a Figure 0b Figure 0C L Figure 1a Figure 1b Figure 1C Shi4

Claims (2)

[Claims] (1) Binarize each image data of pixels of minute divisions that make up the original image to generate binarized data, divide the binarized data into regions with high spatial density, and divide the binarized data into binarized data. An image processing method that divides a texture area based on the weight between the area to be divided based on its spatial continuity and the area with high spatial density of binarized data. (2) The image processing method according to claim 1, wherein the binarized data is obtained by binarizing change data indicating a spatial change in the image data.