JPH09161074A - Picture processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily generate three-dimensional data from a single two-dimensional picture by extracting an outline from a picture and estimating the front and behind relation of areas divided by the outline based upon a part where the outlines intersect each other in a T shape. SOLUTION: A reading means 2 reads out a picture to be processed from a frame buffer 1 as a picture output. A contour extracting means 3 extracts an outline from the picture output read out by the means 2 as a contour output. An intersection detecting means 4 detects a part where the outlines intersect each other in a T shape from the contour output and outputs the coordinates of the intersection and the rotational angle of the T shape obtained by setting the depth direction as a rotation axis. The intersection detection of the means 4 is executed for all areas. Then, an area dividing means 5 divides the picture into plural areas by using the contour output as a boundary. A 1st depth estimating means 6 estimates distance relation between respective areas outputted from the means 5 from the position and angle outputs from the means 4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus for extracting a perspective relationship between a plurality of objects in an image from a two-dimensional image that does not have distance information in the depth direction.

[0002]

2. Description of the Related Art In recent years, as the performance of computers has increased, three-dimensional CGs have become available for personal use computers.
Etc. are becoming easier to handle. However, the shape data of a three-dimensional object is different from the two-dimensional data,
Data acquisition was difficult because there was no device for easy input. In order to solve this, in the field of image recognition, there is an increasing need for a technology for acquiring three-dimensional information such as shape recognition and restoration of a three-dimensional object from a image input from a camera or extraction of distance information in the depth direction. There is.

For example, as a method for obtaining the distance to an object, a method has been devised in which two cameras installed at different positions photograph the same object and the distance is calculated using parallax.

[0004]

However, the above-mentioned configuration requires a special photographing device, and thus an increase in cost cannot be avoided, and moreover, the two-dimensional image data photographed and accumulated in the past is used. It had a problem that it could not. There is also a problem that it is not suitable for real-time processing because all parallaxes of each pixel must be processed.

In view of the above problems, the present invention focuses on the fact that the contours of two objects intersect in a T-shape when a distant object hides a part of a distant object, and no special photographing device is required. Another object of the present invention is to provide an image processing device that generates 3D data by simply estimating / extracting the perspective relationship between objects from any 2D image including images accumulated in the past.

[0006]

In order to solve the above problems, the present invention is an image for generating a three-dimensional data by extracting a perspective relationship between a plurality of objects from a two-dimensional image having no distance information in the depth direction. In the processing device, a frame buffer that stores a two-dimensional image, an image reading unit that reads and outputs the two-dimensional image from the frame buffer, and an output of the image reading unit as an input, and a contour that extracts and outputs a contour line from the input The output of the extraction means and the contour extraction means is used as an input, and the coordinates at which the input contour lines intersect in a T-shape are output as a position, and the T-shape formed by the intersecting contour lines rotates about the depth direction. An intersection point detecting means for outputting an angle as an angle output, an output of the image reading means and an output of the contour extracting means as inputs, and the output of the image reading means as a boundary between the contour lines output by the contour extracting means. Then, the area dividing means for dividing and outputting the area, the position output, the angle output of the intersection detecting means, and the output of the area dividing means are input, and the area dividing means outputs the position output and the angle output of the intersection detecting means. The estimation means estimates the perspective relationship between the regions to be output and outputs it.

According to the present invention, the image reading means first reads the processed image from the frame buffer and outputs the processed image. The contour extracting means extracts a contour from the output of the image reading means and outputs it. The area dividing means divides the image into areas by using the contour line output from the contour extracting means as a boundary and outputs the image. The intersection point detecting means detects the portion where the contour intersects the T-shape from the output of the contour extracting means, and the detected position and inclination of the T-shape are used as the position output and the angle output, respectively. The first estimating means estimates and outputs the perspective relationship between the areas output by the area division output from the position output and the angle output of the intersection detecting means.

According to the present invention, the depth of an object can be estimated from a two-dimensional image to generate three-dimensional data by paying attention to the shape of a contour line generated when an object near the viewpoint blocks an object at a distance. it can.

In the second invention, in order to improve the accuracy of the generated three-dimensional data, the relative distance from the viewpoint of the micro area calculated based on the fineness of the pattern for each micro area is used.

In the third invention, the relative distance from the viewpoint obtained from the density of the contour line is used to improve the accuracy. Since the amount of processing increases if the fineness of the pattern is directly handled, by substituting the fineness of the pattern with the density of the contour line, an image processing apparatus that improves the accuracy of three-dimensional data and suppresses the increase in the amount of processing is provided. It can be realized.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image processing apparatus according to an embodiment of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention.

In FIG. 1, reference numeral 1 is a frame buffer in which an image to be processed is stored. Reference numeral 2 is a reading means for reading the image to be processed from the frame buffer 1 and outputting the image. Reference numeral 3 is a contour extracting means that receives the image output of the reading means 2 and extracts a contour line from the input to output the contour. Reference numeral 4 denotes an intersection point detecting means which receives the contour output of the contour extracting means 3 as an input, detects an angle at which the input contour lines intersect with a coordinate at which they intersect in a T-shape, and outputs the coordinate as a position and the angle as an angle output. . Reference numeral 5 denotes a region dividing unit which receives the contour output of the contour extracting unit 3 as an input and divides the image into regions by using the contour output of the contour extracting unit 3 as a boundary and outputs the region. 6 receives the position output and the angle output of the intersection point detecting means 4 and the area output of the area dividing means 5 and inputs the position output and the angle output of the intersection point detecting means 4 between the divided areas output by the area dividing means 5. It is a first estimating means for estimating a perspective relationship and producing a first estimation output.

The operation of the image processing apparatus configured as described above will be described below with reference to FIG. However,
Here, the image to be processed is the image shown in FIG. FIG. 5 shows the output of each unit in the image processing process according to this embodiment.

First, the reading means 2 reads the image to be processed from the frame buffer 1 and outputs it as an image (see FIG. 4).

The contour extracting means 3 extracts a contour line from the image output of the reading means 2 and uses it as a contour output (see FIG. 5 (a)).

Next, the intersection detecting means 4 detects a portion where the contour lines intersect in a T-shape from the contour output of the contour extracting means 3, and the coordinate of the intersection and the T-shape rotate about the depth direction as a rotation axis. Is output. For example, if the xy coordinate of the intersection is (100,50) and the rotation of the T-shape is 45 ° in the counterclockwise direction, the position output of the intersection detection means 4 is (100,50) and the angle output is 45 °. . The intersection detection of the intersection detection means 4 is performed for the entire input area, and in the case of the image to be processed of this embodiment, position output (100, 50), angle output 45 °, position output (100, 150),
Two sets of angle output -45 ° (see FIG. 5 (b)) are the intersection point detecting means 4.
Output.

The area dividing means 5 divides the image into areas by using the contour output of the contour extracting means 3 as a boundary. In the case of the image to be processed of this embodiment, the image output is divided into three areas and becomes area output (see FIG. 5 (c)).

The first depth estimating means 6 is the area dividing means 5.
The inferior-far relationship between the areas output by is estimated from the position output and the angle output of the intersection detecting means 4. In the case of the image to be processed according to the present embodiment, the intersection detecting means 4 outputs two sets of outputs, so that the estimation by the first depth estimating means 6 is also performed twice.

The first depth estimating means 6 outputs the position output (100, 5
0), the angle output 45 ° is received, the coordinates (100,5
In (0), it is interpreted that the contour line extending from the lower right 45 ° is cut off by the contour line rising to the right, and from this, the area A at the upper left of the coordinates (100,50) is divided into area B and area C. It is presumed that it exists in a position that blocks the light. That is, it is interpreted that the area A exists closer to the viewpoint than the areas B and C.

Similarly, position output (100, 150), angle output-45
It is presumed that the region A is closer to the viewpoint than the region B is because the region A is nearer to the region B and the region C than the input of °. The result is obtained and used as the first guess output.

Further, if a contradiction occurs between a plurality of inference results, a majority decision is made and the inference result that minimizes the number of inconsistent inferences is used as the first inference output, or The estimation result that minimizes the value obtained by multiplying the number of estimations by the area of the contradictory area is selected as the first depth estimation output.

As described above, according to the present embodiment, the image in which the perspective information is extracted from a single two-dimensional image by paying attention to the position where the contour intersects the T-shape and the inclination of the T-shape forming the contour. A processing device can be provided.

(Second Embodiment) FIG. 2 is a block diagram of an image processing apparatus according to a second embodiment of the present invention. 2, those having the same functions as those in FIG. 1 are designated by the same reference numerals, and detailed description thereof will be omitted. However, here, the image to be processed is the image shown in FIG. FIG. 6 shows the output of each unit in the image processing process according to this embodiment.

In FIG. 2, reference numeral 7 is a first distance detecting means 7 which receives the image output of the image input means 1, divides the input into minute blocks of a certain size, and estimates the distance of the minute blocks and outputs the estimated distance. Is.

Reference numeral 71 denotes a first block dividing means 71 which divides an input into minute blocks of a certain size to obtain a first block output (see FIG. 6 (d)). Reference numeral 72 is a frequency conversion means 72 which receives the first block output of the first block dividing means 71 as an input, and performs a two-dimensional frequency conversion of the input block to generate a frequency output. 73 is frequency conversion means 7
The pattern detecting means 73 receives the frequency output of 2 as an input, detects the fineness of the pattern based on the frequency distribution of the input, and outputs the detected output (see FIG. 6E). Reference numeral 74 denotes a first distance calculation unit 74 which receives the detection output of the pattern detection unit 73 and calculates and outputs the block distance based on the fineness of the input pattern (see FIG. 6 (f)).

Numeral 8 receives the first depth estimation output of the estimating means 6 and the output of the first distance detecting means 7 as inputs, and estimates the combination of the perspective relationships having the least contradiction with respect to both inputs and outputs the second. It is the second depth estimation means 8 which outputs an estimated output.

As described above, according to this embodiment, attention is paid to the position where the contour intersects the T-shape and the inclination of the T-shape forming the contour, and at the same time to the fineness of the pattern for each small area of the two-dimensional image. By doing so, it is possible to provide an image processing device that accurately extracts perspective information from a single two-dimensional image.

(Third Embodiment) FIG. 3 is a block diagram of an image processing apparatus according to a third embodiment of the present invention. In FIG. 3, those having the same functions as those in FIG. 1 are designated by the same reference numerals, and detailed description thereof will be omitted. However, here, the image to be processed is the image shown in FIG. FIG. 7 shows the output of each unit in the image processing process according to this embodiment.

In FIG. 3, reference numeral 9 designates the contour output of the contour extracting means 2 as an input, and after dividing the input into minute blocks of a certain size, estimates the distance of the minute blocks and outputs the second.
The distance detecting means 9 of FIG.

Reference numeral 91 is a second block dividing means 91 for dividing an input into minute blocks of a fixed size to obtain a second block output (see FIG. 7 (d)). Reference numeral 92 denotes a density detection / detection unit 92 which receives the second block output of the second block dividing unit 92 as an input, detects the density of the contours in the input block and outputs the density as a density output (see FIG. 7 (e)). is there. 93
Is a second distance calculating means 93 which receives the density output of the density detecting means 92, estimates the distance of the block based on the input, and outputs the estimated distance (see FIG. 7 (f)).

Numeral 10 receives the first depth estimation output of the first estimation means 6 and the output of the second distance detection means 9 as inputs, and estimates a combination of perspective relations having the least contradiction for both inputs. Third depth estimation means 10 for estimating output
It is.

The difference from FIG. 2 is that instead of using the frequency conversion of the fineness of the pattern for estimating the distance, the detection of the contour density is used. As a result, the structure of the detection means is simplified and the detection speed is increased.

[0034]

As described above, the present invention provides means for extracting a contour line from an image and estimating the front-rear relationship between regions separated by the contour line based on the portion where the contour line intersects in a T shape. By providing the image processing apparatus, it is possible to provide an image processing apparatus that generates three-dimensional data from a single two-dimensional image without relying on any special photographing apparatus or photographing technique.

[Brief description of the drawings]

FIG. 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram of an image processing apparatus according to a second embodiment of the present invention.

FIG. 3 is a block diagram of an image processing apparatus according to a third embodiment of the present invention.

FIG. 4 is a diagram showing an image to be processed for explaining the first embodiment of the present invention.

FIG. 5 is an output diagram of an image processing process according to the first embodiment of the present invention.

FIG. 6 is an output diagram of an image processing process according to the second embodiment of the present invention.

FIG. 7 is an output diagram of an image processing process according to a third embodiment of the present invention.

[Explanation of symbols]

 1 frame buffer 2 image reading means 3 contour extracting means 4 intersection detecting means 5 area dividing means 6 first depth information estimating means 7 first distance detecting means 71 first block dividing means 72 frequency converting means 73 pattern detecting means 74 First distance calculating means 8 Second depth information estimating means 9 Second distance detecting means 91 Second block dividing means 92 Density detecting means 93 Second distance calculating means 10 Third depth information estimating means

Claims (3)

[Claims] 1. An image processing apparatus for generating a three-dimensional data by extracting a perspective relationship between a plurality of objects from a two-dimensional image having no distance information in the depth direction; a frame buffer for storing the two-dimensional image; An image reading means for reading and outputting an image from a buffer; a contour extracting means for inputting the output of the image reading means and extracting a contour line from the output of the image reading means for output; and an output of the contour extracting means. From the output of the contour extracting means, the portion where the contour lines intersect in a T-shape is detected from the output of the contour extraction device, and the intersecting coordinates are output as a position. The T-shape formed by the intersecting contour lines is tilted about the depth direction. The intersection point detecting means for making the angle output as an angle output, and the output of the contour extracting means as an input, and the contour line image as a plurality of regions with the output of the contour extracting means as a boundary. Area dividing means for dividing and outputting, and the position output and angle output of the intersection point detecting means and the output of the area dividing means as input, and the area dividing means based on the position output and angle output of the intersection point detecting means Estimate the perspective relationship between multiple output regions and use this as depth data.
An image processing apparatus, comprising: 2. The input of the output of the image reading means,
A first block dividing unit that divides the output of the image reading unit into blocks of a certain size for output, and an output of the first block dividing unit as an input, and an output of the first block dividing unit as a block Frequency conversion means for performing two-dimensional frequency conversion for each output and pattern detection means for receiving the output of the frequency conversion means as input and detecting the fineness of the pattern for each block from the output of the frequency conversion means for output A first distance calculation in which the output of the pattern detection means is input, and the distance in the depth direction for each block is calculated and output from the output of the pattern detection means under the assumption that the finer the pattern is, the farther away it is. Means, the first estimation output of the first estimation means and the output of the first distance calculation means, and the first estimation output of the first estimation means and the first distance calculation. When there is no contradiction between the outputs of the output means, the output of the first distance calculation means is directly used as the second estimated output, and when there is a contradiction, the output of the first distance calculation means with respect to the contradiction point. 2. The image processing apparatus according to claim 1, further comprising: a second estimation unit that outputs a second estimation output that is a combination of perspective relations with which the contradiction is minimized. 3. A second block division means for receiving the output of the contour extraction means as an input and dividing the output of the contour extraction means into blocks of a certain size for output, and an output of the second block division means. , The density detection means for detecting and outputting the density of the contour line in the block output by the second block dividing means, and the output of the density detection means as an input, Second distance calculating means for calculating and outputting a distance in the depth direction of the block from the output of the density detecting means under the assumption of being distant, a first estimated output of the first estimating means, and the above When the output of the second distance calculation means is input, and there is no contradiction between the first estimated output of the first estimation means and the output of the second distance calculation means, the second distance calculation is performed. The output of the means as it is 3rd inference, and if there is a contradiction, the combination of the perspective relations with which the contradiction is the smallest with respect to the output of the second distance calculation means is used as the 3rd inference output for the inconsistent part. The image processing apparatus according to claim 1, further comprising: