JP2012234432A - Vanishing point calculation device, vanishing point calculation method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vanishing point calculation device capable of stably calculating a vanishing point even when many straight lines are included in an image in addition to straight lines passing through the vanishing point.SOLUTION: Images are prepared as plural learning images, Global feature amounts are calculated from the learning images based on each category, and a discriminator is previously created by leaning the feature amounts. Scene constitution estimation means 21 calculates feature amounts from a taken input image and inputs the resultant in a previously created scene constitution discriminator, and estimates a vanishing point existing candidate area which is a candidate area where a vanishing point may exist based on the category output from the scene constitution discriminator. Vanishing point calculation means 22 detects only straight lines that pass through the vanishing point existing candidate area as a straight line group from an image which has been subjected to a straight line detection on the input image, and determines an intersection point of two major straight lines from the straight line group as the vanishing point.

Description

  The present invention relates to a vanishing point calculation device for calculating a vanishing point for generating depth estimation data from a single photographed image that does not have depth information obtained by photographing a building or a scene composed of planes such as in a building, The present invention relates to a point calculation method and a program.

  Conventionally, a building is assumed to be a rectangular parallelepiped from image data of a single photograph as shown in FIG. 7A, edge extraction is performed as shown in FIG. 7B, and three-dimensional reconstruction is performed from the position of the vanishing point. There is a method (see, for example, Non-Patent Document 1).

IPSJ Research Report. CVIM, "Statistical optimization of 3D reconstruction from a single image", 2004 (113), 117-124, 2004-11-11

  As described above, as shown in FIG. 7A, the image 100 taken indoors conceals a plane that forms a building such as a ceiling, air conditioning of a wall, a wall, or goods in a store. As shown in FIG. 7B, it is difficult to distinguish an edge that forms a vanishing point from a plurality of edges, and it is difficult to obtain a correct vanishing point, as shown in FIG. 7B. There is a problem that.

  The present invention has been made in consideration of such circumstances, and its purpose is to stably calculate the vanishing point even if a number of straight lines other than the straight line passing through the vanishing point are included in the image. A vanishing point calculating device, a vanishing point calculating method, and a program.

  In order to solve the above-described problem, the present invention is a vanishing point calculation device that calculates a vanishing point from a single image of a building or the like, and creates a category for each vanishing point position for learning. A learning image category classification means for classifying the acquired plurality of captured images for each category, and feature quantities are extracted from the plurality of learning images classified into categories by the learning image category classification means, and characteristics of the feature quantities are obtained. A scene composition discriminator creating means for creating a scene composition discriminator by learning, an image photographing means for photographing an input image composed of planes, and extracting feature quantities from the input image photographed by the image photographing means. , A scene configuration in which the feature amount is input to the scene configuration discriminator, and a vanishing point existence candidate region in which a vanishing point exists is estimated based on a category output from the scene configuration discriminator A constant section, on the basis of the vanishing point existing candidate region estimated by the scene estimating means, a vanishing point computing device, characterized in that it comprises a vanishing point calculation means for calculating a vanishing point.

  According to the present invention, in the above invention, the scene configuration estimation unit includes a feature amount calculation unit that calculates a feature amount for each category from an image having a plurality of resolutions, and a feature amount calculated by the feature amount calculation unit. Learning means for creating a discriminator by learning.

  The present invention is the above invention, wherein the vanishing point calculation means detects only a straight line passing through the vanishing point existence candidate region as a straight line group from an image obtained by performing predetermined image processing on the input image, Work to take two straight lines from the straight line group, find the intersection of the two straight lines, calculate a straight line passing through the cross point from the straight line group, and use the number as the number of votes of the intersection Is repeated while randomly selecting two straight lines, and the intersection with the largest number of votes is the vanishing point.

  Further, in order to solve the above-described problem, the present invention is a vanishing point calculation method for calculating a vanishing point from a single image obtained by photographing a building or the like. A learning image category classification step for classifying a plurality of captured images acquired for each category, and extracting feature amounts from the plurality of learning images classified into categories in the learning image category classification step, A scene composition discriminator creating step for creating a scene composition discriminator by learning characteristics, an image photographing step for photographing an input image composed of faces, and a feature amount from the input image photographed in the image photographing step Extract and input the feature quantity to the scene configuration discriminator, and estimate the vanishing point existence candidate region where the vanishing point exists based on the category output from the scene configuration discriminator That the scene structure estimation step, on the basis of the vanishing point existing candidate region estimated by the scene estimation step, a vanishing point calculation method which comprises a vanishing point calculation step of calculating a vanishing point.

  According to the present invention, in the above invention, the scene configuration estimation step includes a feature amount calculation step for calculating a feature amount for each category from an image having a plurality of resolutions, and a feature amount calculated by the feature amount calculation step. And a learning step for creating a discriminator.

  According to the present invention, in the above invention, the vanishing point calculating step detects only a straight line passing through the vanishing point existence candidate region as a straight line group from an image obtained by performing predetermined image processing on the input image, Work to take two straight lines from the straight line group, find the intersection of the two straight lines, calculate a straight line passing through the cross point from the straight line group, and use the number as the number of votes of the intersection Is repeated while randomly selecting two straight lines, and the intersection with the largest number of votes is the vanishing point.

  In order to solve the above-described problem, the present invention creates a category for each vanishing point position in a computer of a vanishing point calculation device that calculates a vanishing point from a single image of a building or the like, and learns it. A learning image category classification function for classifying a plurality of captured images acquired for each category, a feature amount is extracted from a plurality of learning images classified into categories by the learning image category classification function, and characteristics of the feature amount are extracted A scene composition discriminator creating function for creating a scene composition discriminating function by learning the above, an image photographing function for photographing an input image composed of faces, and extracting feature quantities from the input image photographed by the image photographing function, The feature amount is input to the scene configuration determination function, and a scene configuration for estimating a vanishing point existence candidate region where a vanishing point exists is based on the category output from the scene configuration determination function. Estimating function, on the basis of the vanishing point existing candidate region estimated by the scene estimation function, a program characterized by executing the vanishing point calculation function of calculating a vanishing point.

  According to the present invention, the vanishing point can be calculated stably even if a large number of straight lines other than the straight line passing through the vanishing point are included in the image.

It is a conceptual diagram for demonstrating the example of a category by this embodiment. It is a block diagram which shows the structure of the scene structure discrimination device by this embodiment. It is a block diagram which shows the structure of the vanishing point calculation apparatus for calculating a vanishing point from an input image by this embodiment. It is a schematic diagram which shows the example of a category when two vanishing points are contained in the image. It is a conceptual diagram for demonstrating the feature-value for creating the scene structure discriminator by this embodiment. In this embodiment, it is a conceptual diagram for demonstrating the vanishing point calculation method. It is a conceptual diagram explaining the factor which makes it difficult to calculate a vanishing point in a prior art.

  There is a learning image in which the position of the vanishing point is known. Divide images into categories according to vanishing point positions. A discriminator for identifying a category is created using the feature amount of the image. For an actual input image, a category is determined by using a discriminator. If the category is determined, the vanishing point existence candidate region is known. A point (voting process) with many straight line intersections limited to the candidate area is defined as a vanishing point.

  More specifically, a plurality of locations of the planar structure targeted in advance are photographed from various directions, and categorization is performed based on the photographing direction or the location of the vanishing point on the image. For example, as shown in FIGS. 1A to 1C, the planar structure is divided into three categories 201, 202, and 203 that are assumed when the image is taken from the passage in the building toward the ceiling, and the captured images of the respective patterns. Are prepared as learning images. A global feature value is calculated from the learning image for each category, and the feature value is learned to create a discriminator. The category to which the input image belongs is estimated by applying to the classifier, and the position of the vanishing point (vanishing point existence candidate region) is estimated.

  When calculating the vanishing point, it is possible to narrow down only to a straight line passing through the vanishing point existence candidate region. As a result, the straight line forming the vanishing point can be extracted more accurately, and the position of the vanishing point can be accurately calculated. As described above, before calculating the vanishing points, by estimating or not the input image is obtained by photographing the location of any planar structure in advance, linear non-linear with many single input image to form a vanishing point Even from this point, the accuracy of the vanishing point position can be maintained to some extent. In addition, by this, it is possible to estimate the three-dimensional position in the image using the vanishing point and generate a pseudo three-dimensional image, and of course, perform segmentation of the image using the three-dimensional position estimated value to obtain a desired region. It is also possible to extract.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 2 is a block diagram showing the configuration of the scene configuration discriminator according to this embodiment. In FIG. 2, the scene configuration discriminator 10 includes a learning image category classification unit 11 and a scene configuration discriminator creation unit 12 that categorize a plurality of learning images.

  The learning image category classification unit 11 first determines a category, and prepares a plurality of learning images classified into each category. A category is created by classifying a target location such as a passage in an underground shopping street for each shooting direction or each vanishing point location. For example, description will be made based on examples of three categories in which a ceiling and a wall are photographed from a passage in an underground shopping street as shown in FIGS.

  In FIG. 1 (a), 201 is a scene that is photographed in front of the direction of the passage, and in FIG. 1 (b), 203 is a scene that includes a plane perpendicular to the ground on the right side and a ceiling. In FIG. 1C, 202 can be divided into three categories such as a scene composed of a plane perpendicular to the ground on the left side and the ceiling. These three categories are classified according to the vanishing point existing area.

  The category 201 is divided when the x-coordinate of the vanishing point is in the region 204, the category 202 is divided into the x-coordinate of the vanishing point in the region 205, and the category 203 is divided when the x-coordinate of the vanishing point is in the region 206. However, the present invention is not limited to this, and other scenes such as a scene with a floor can be learned. In FIGS. 1A to 1C, although one vanishing point is classified for each photographing direction, the vanishing points may be classified for each number. For example, as an example of two vanishing points, a scene having two vanishing points 501 and 502 may be added to the category as shown in FIG.

  Next, the scene configuration discriminator creating means 12 will be described. The scene configuration discriminator creating unit 12 includes a feature amount calculating unit 121 and a learning unit 122 that learns the feature amount calculated by the feature amount calculating unit 121 and creates a discriminator.

  The feature amount calculation unit 121 first creates images with a plurality of resolutions. Next, an image obtained by applying a Gaborfilter in each direction (vertical, horizontal, and diagonal directions) of the input image 301 shown in FIG. 5A [Reference 1: JG Daugman: Uncertainty relations for resolution in space, spatial frequency, and orientation optimized. by two-dimensional visual cortical filters, Journal of the Optical Society of America A, 1985, vol. 2, pp. 1160-1169.], as shown in Fig. 5 (b). An image 302 is created.

  In the case where the image (Gabor image) 302 to which the Gaborfilter is applied is binarized, the number of pixels having a pixel value of 1 is counted in each of the 16 divided regions. If the Gabor image 302 is a 256-color image corresponding to the edge strength as shown in the figure, the pixel value may be counted. Alternatively, an average value of each of the 16 divided areas is taken. The above processing is performed at each resolution. For example, when two resolutions of 1/4 and 1/10 are used, 2 (resolution) × 16 (region / sheet) × 4 (differential component of direction) = 128 dimensions for one input image This is a feature value.

  Although the method for calculating the global feature value has been described, any feature value may be used as long as the scene configuration can be expressed.

  Next, the learning unit 122 uses the neural network (reference document 2) to display the feature amount calculated by the feature amount calculation unit 121 and the teacher data representing each category as shown in FIGS. : “Pattern recognition and learning statistics” Iwanami Shoten pp.23-31 2005 5th edition). The discriminator to be learned is not limited to a neural network, and any class such as multi-class SVM (Reference 3: “Statistics of Pattern Recognition and Learning” Iwanami Shoten pp.107-118 2005 5th Edition) may be used. .

  Next, a method for calculating the vanishing point using the scene configuration discriminator created by the above-described method will be described.

  FIG. 3 is a block diagram illustrating a configuration of a vanishing point calculation device for calculating a vanishing point from an input image according to the present embodiment. FIG. 6 is a conceptual diagram for explaining a vanishing point calculation method in the present embodiment.

  In FIG. 3, the image capturing means 20 captures an image 401 as shown in FIG. The scene configuration estimation unit 21 calculates the feature amount from the input image 401 captured by the image capturing unit 20 in the same manner as the Gabor image 302 illustrated in FIG. 5B, and the feature amount is determined as the scene configuration determination illustrated in FIG. The vanishing point existence candidate region 402, which is a candidate region where the x coordinate of the vanishing point exists, is estimated, as shown in FIG. 6B, in accordance with the category input to the device 10 and output from the scene configuration discriminator 10. To do.

  As shown in FIG. 6C, the vanishing point calculating means 22 detects only a straight line passing through the vanishing point existence candidate region 402, etc. (Reference 4: “OpenCV” programming book Mainichi Communications 2007 pp.164). -167) is detected as a straight line group 403, two straight lines are randomly extracted from the straight line group 403, and an intersection of the two straight lines is obtained. A straight line passing through the intersection is calculated from the straight line group 403, and the number is obtained as the number of votes obtained at the intersection. The above operation is repeated while randomly selecting two straight lines. The intersection with the largest number of votes is defined as the vanishing point 404.

  The vanishing point 404 may be obtained by such a method, or the length is regarded as the reliability, and the intersection of the longest straight line and the second longest straight line in the detected straight line group 403 is vanished. Any method may be used such as determining a point.

  According to the above-described embodiment, a straight line other than the straight line passing through the vanishing point is obtained by learning the scene configuration based on the area where the vanishing point exists in advance from an image obtained by photographing a scene having a plurality of structures. Even if it is included in a large number of images, the vanishing point can be calculated stably. Thereby, three-dimensional information can be estimated from one non-stereo image using a vanishing point. In addition to generating a pseudo stereoscopic image, it is possible to perform segmentation of an image using depth data and extract a desired region.

DESCRIPTION OF SYMBOLS 10 Scene structure discriminator 11 Learning image category classification | category means 12 Scene structure discriminator creation means 121 Feature-value calculation means 122 Learning means 20 Image photographing means 21 Scene structure estimation means 22 Vanishing point calculation means

Claims (7)

A vanishing point calculating device that calculates a vanishing point from a single image of a building,
A category for each vanishing point position, and a learning image category classification means for classifying a plurality of captured images acquired for learning for each category,
Scene configuration discriminator creating means for extracting a feature quantity from a plurality of learning images classified into categories by the learning image category classification means, and creating a scene configuration discriminator by learning the characteristics of the feature quantity;
Image photographing means for photographing an input image composed of surfaces,
A feature amount is extracted from an input image photographed by the image photographing means, the feature amount is input to the scene configuration discriminator, and a vanishing point exists based on a category output from the scene configuration discriminator. A scene configuration estimating means for estimating a point presence candidate area;
A vanishing point calculating unit comprising: a vanishing point calculating unit that calculates a vanishing point based on the vanishing point existence candidate region estimated by the scene estimating unit. The scene configuration estimation means includes
A feature amount calculating means for calculating a feature amount for each category from images of a plurality of resolutions;
The vanishing point calculation apparatus according to claim 1, further comprising: a learning unit that learns the feature amount calculated by the feature amount calculation unit and creates a discriminator. The vanishing point calculating means is
Only a straight line passing through the vanishing point existence candidate region is detected as a line group from an image obtained by performing predetermined image processing on the input image, and two lines are randomly extracted from the line group, The process of obtaining the intersection of straight lines, calculating the straight line passing through the intersection from the group of straight lines, and setting the number as the number of votes of the intersection is repeated while randomly selecting two straight lines. The vanishing point is the intersection of the most
The vanishing point calculation apparatus according to claim 1 or 2, wherein A vanishing point calculation method for calculating a vanishing point from a single image of a building,
A learning image category classification step for creating a category for each vanishing point position and classifying a plurality of captured images acquired for learning for each category;
A scene configuration discriminator creating step of extracting a feature amount from a plurality of learning images classified into categories in the learning image category classification step and creating a scene configuration discriminator by learning characteristics of the feature amount;
An image capturing step for capturing an input image composed of surfaces;
A feature amount is extracted from the input image captured in the image capturing step, the feature amount is input to the scene configuration discriminator, and a vanishing point exists based on a category output from the scene configuration discriminator. A scene configuration estimation step for estimating a point presence candidate region;
A vanishing point calculating step of calculating a vanishing point based on the vanishing point existence candidate region estimated in the scene estimating step. The scene configuration estimation step includes:
A feature amount calculating step for calculating a feature amount for each category from images of a plurality of resolutions;
The vanishing point calculation method according to claim 4, further comprising a learning step of learning the feature amount calculated in the feature amount calculation step to create a discriminator. The vanishing point calculating step includes:
Only a straight line passing through the vanishing point existence candidate region is detected as a line group from an image obtained by performing predetermined image processing on the input image, and two lines are randomly extracted from the line group, The process of obtaining the intersection of straight lines, calculating the straight line passing through the intersection from the group of straight lines, and setting the number as the number of votes of the intersection is repeated while randomly selecting two straight lines. The vanishing point is the intersection of the most
The vanishing point calculation method according to claim 4, wherein the vanishing point is calculated. To the computer of the vanishing point calculation device that calculates the vanishing point from a single image of a building etc.,
Image category classification function for learning that creates a category for each vanishing point position and classifies a plurality of captured images acquired for learning for each category,
A scene configuration discriminator creating function for creating a scene configuration discriminating function by extracting feature amounts from a plurality of learning images classified into categories by the learning image category classification function and learning characteristics of the feature amounts;
Image capture function for capturing input images composed of
A feature amount is extracted from an input image captured by the image capturing function, the feature amount is input to the scene configuration determination function, and a vanishing point exists based on a category output from the scene configuration determination function. Scene composition estimation function for estimating point existence candidate areas,
A program for executing a vanishing point calculation function for calculating a vanishing point based on the vanishing point existence candidate region estimated by the scene estimation function.

Images