KR101260132B1 - Stereo matching process device, stereo matching process method, and recording medium - Google Patents

Abstract

The image data input unit 10 acquires image data of a plurality of images obtained by photographing a predetermined area from a plurality of different positions. The reference parallax setting unit 12 sets the standard parallax corresponding to the plurality of images. The search range setting unit 13 sets a predetermined range smaller than the range of the image as the search range for stereo matching based on the point of the image giving the parallax of the reference set by the reference parallax setting unit 12. The stereo matching unit 11 is a search range setting unit (based on a point of another image which gives a parallax of a reference set by the reference parallax setting unit 12 with respect to an arbitrary point in one image of a plurality of images. In the search range set in 13), a corresponding point of another image is searched for.

Description

STEREO MATCHING PROCESS DEVICE, STEREO MATCHING PROCESS METHOD, AND RECORDING MEDIUM}

The present invention relates to a stereo matching processing apparatus, a stereo matching processing method and a computer readable recording medium. More specifically, the present invention relates to a method for automatically generating three-dimensional data from a stereo image.

As a method of automatically generating three-dimensional data from a stereo image, a method of generating three-dimensional data (Digital Surface Model) data representing a terrain by stereo matching based on an image obtained from a satellite or an aircraft is widely used. It is done. Here, with respect to two images taken at a different point in time from the stereo matching process, a so-called stereo image, a corresponding point in each image picking up the same point is obtained, and the parallax is used to reach the object by the principle of triangulation. It is to find the depth or shape.

Various methods have already been proposed for this stereo matching process. For example, Patent Document 1 discloses a method using an area correlation method which is generally used. This area correlation method is a method of obtaining a correspondence point by setting a correlation window in a left image and making it a template, moving the search window in the right image to calculate a cross correlation coefficient with the template, and considering this as a degree of agreement and searching for a high point.

In the above method, in order to reduce the throughput, by limiting the moving range of the search window to the epipolar line direction in the image, the position shift amount in the x direction of the point in the corresponding right image, i.e. parallax, is reduced with respect to each point in the left image. You can get it. Here, the epipolar line is a straight line which can be drawn as an existence range of a point corresponding to the point in the other image with respect to any point in one image in the stereo image. The epipolar line is described in the "Image Analysis Handbook" (supervised by Takasaki Kokusai, supervised by Tokyo University Press, January 1991, pages 597-599).

Usually, the epipolar line direction is different from the scanning line direction of the image. However, by changing the coordinates, rearrangement can be performed by matching the epipolar line direction to the scanning line direction of the image. The method of this coordinate change is described in said "image analysis handbook". In the rearranged stereoscopic image, the movement range of the search window of the corresponding point can be limited to the scanning line. Therefore, parallax is obtained as the difference of the x coordinate of the correspondence point in a left-right image.

In the stereo image, portions (occluded regions), which become shadows of objects, respectively occur. By not giving the stereo matching correspondence to this occlusion area, the matching method which gives an accurate correspondence is proposed (refer patent document 2).

In addition, Patent Document 3 describes a technique of a stereo image processing apparatus in which three-dimensional data is automatically obtained for a complicated object from a satellite stereo image or an aerial stereo image without passing through an operator. In the technique of Patent Document 3, incorrect data such as noise or deficiency in the three-dimensional data obtained by the stereo processing means is automatically corrected using the appearance information such as a building obtained from the map data of the map data storage means. The map data accumulating means provides map data such as the appearance of the building to the DSM data automatic correction means.

Patent Document 1: Japanese Patent Application Laid-open No. 03-167678 Patent Document 2: Japanese Patent Application Laid-Open No. 04-299474 Patent Document 3: Japanese Patent Application Laid-Open No. 2002-157576

In one set of images performing stereo matching processing, when searching for a corresponding point of another image with respect to an arbitrary point in one image, the process is to search over the entire scanning line (epipolar line) of the image. Takes time, and also increases the probability of miss matching.

This invention is made | formed in view of the above-mentioned situation, and an object of this invention is to improve the speed and precision of a stereo matching process.

The stereo matching processing device according to the first aspect of the present invention,

An image data acquisition unit for acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;

A reference parallax setting unit that sets a parallax of a reference corresponding to the plurality of images;

A search range setting unit for setting a predetermined range smaller than the range of the image as a search range for stereo matching on the basis of the point of the image giving the parallax of the reference set by the reference parallax setting unit;

With respect to an arbitrary point in one image of the plurality of images, the other image in the search range set by the search range setting unit is based on the point of another image which gives the parallax of the reference set by the reference parallax setting unit. Search unit for searching the corresponding point

Characterized in having a.

Stereo matching processing method according to a second aspect of the present invention,

An image data acquisition step of acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;

A reference parallax setting step of setting a standard parallax corresponding to the plurality of images;

A search range setting step of setting a predetermined range smaller than the range of the image data as a search range for stereo matching, based on the point of the image giving the parallax of the reference set in the reference parallax setting step;

With respect to any point in one image of the plurality of images, the other image in the search range set in the search range setting step is based on the point of another image which gives the parallax of the reference set in the reference time difference setting step. Search step for searching for corresponding point

And FIG.

A computer-readable recording medium that records a program according to the third aspect of the present invention,

Computer,

An image data acquisition unit for acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;

A reference parallax setting unit that sets a parallax of a reference corresponding to the plurality of images;

A search range setting unit for setting a predetermined range smaller than the range of the image as a search range for stereo matching on the basis of the point of the image giving the parallax of the reference set by the reference parallax setting unit;

With respect to an arbitrary point in one image of the plurality of images, the other image in the search range set by the search range setting unit is based on the point of another image which gives the parallax of the reference set by the reference parallax setting unit. Search unit for searching the corresponding point

It is characterized by recording a program for functioning as.

According to the present invention, the speed and precision of the stereo matching process can be improved in the method for automatically generating three-dimensional data from the stereo image.

1 is a block diagram showing an example of the configuration of a stereo image processing apparatus according to a first embodiment of the present invention.
2 is a diagram schematically showing an example of an aerial photograph converted to image data;
3 is a schematic diagram showing an example of a ground state in the real world.
4 is a schematic diagram showing DSM data generated by stereo matching processing from an image of a part of the real world shown in FIG. 3; FIG.
5 shows a search plane for explaining a search range.
6 illustrates an example of a search range.
7 is a flowchart showing an example of the operation of the height facial expression processing according to the first embodiment.
8 is a block diagram showing a configuration example of a stereo image processing device according to a second embodiment of the present invention.
9 is a diagram illustrating an example of a search plane for setting an elevation of a reference and a search range for each divided region;
10 is a flowchart showing an example of the operation of the height facial expression processing according to the second embodiment.
11 is a block diagram showing a configuration example of a stereo image processing apparatus 1 according to the third embodiment.
12 is a flowchart showing an example of the operation of the height facial expression processing according to the third embodiment.
13 is a block diagram showing a configuration example of a stereo image processing apparatus 1 according to the fourth embodiment.
14 is a flowchart showing an example of the operation of the height facial expression processing according to the fourth embodiment.
FIG. 15 is a block diagram showing a configuration example of a stereo image processing apparatus 1 according to the fifth embodiment.
16 is a flowchart showing an example of the operation of the height facial expression processing according to the fifth embodiment.
17 is a block diagram showing an example of a physical configuration when the stereo image processing apparatus 1 is mounted on a computer.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described in detail, referring drawings. In addition, the same code | symbol is attached | subjected to the same or corresponding part in drawing.

(Embodiment 1)

1 is a block diagram showing a configuration example of a stereo image processing apparatus according to a first embodiment of the present invention. The stereo image processing apparatus 1 includes an image data input unit 10, a stereo matching unit 11, a reference parallax setting unit 12, and a search range setting unit 13. The stereo image processing apparatus 1 is connected to the height calculation unit 2.

The image data input unit 10 has a function of inputting image data, and inputs a plurality of image data used for stereo matching processing. Image data is an image of the aerial photograph converted into a digital image, for example. The image data includes the position where the image was taken, the direction in which the image was taken, the angle of view, and the like.

2 schematically shows an example of an aerial photograph converted to image data. The aerial photograph shown in FIG. 2 is composed of an aerial photograph 101A and an aerial photograph 101B which are continuously photographed from an aircraft flying in the air. The aerial photograph 101A and the aerial photograph 101B were photographed with a 60% overlap with respect to the traveling direction of the aircraft. An overlap part is an image which image | photographed the same area from a different position.

The image in this embodiment is an image generated by digitally converting an aerial photograph including the aerial photograph 101A and the aerial photograph 101B. The images used in the present invention are not limited to aerial images, but images of digitalized satellite photographs, digital images photographed with a general digital camera, digital images obtained by digitizing analog photographs photographed with a general analog camera, etc. It may be.

The stereo matching part 11 of FIG. 1 detects the position in the image which image | photographs the same point with respect to the some image data which image | photographed the same area | region from different positions. That is, a pair of points corresponding to the same points of the plurality of images of the stereo matching unit are detected. The pair of points corresponding to the same point usually takes an image correlation of a corresponding small area among two images, and is detected from a position where the correlation coefficient becomes maximum.

In addition, there are various methods for performing stereo matching processing, such as obtaining and correlating general feature quantities or obtaining correlations between left and right images, but the method used for the stereo matching processing in the present embodiment is limited. none. For example, you may use the stereo matching process of Unexamined-Japanese-Patent No. 08-16930.

The height calculation part 2 produces | generates DSM data by the principle of triangulation from the parallax of the corresponding point obtained by the stereo matching process. For example, since the position of the corresponding feature has a predetermined position shift (parallax) between the aerial photograph 101A and the aerial photograph 101B, the stereo matching process measures this position shift, The height of the surface layer of the feature including the elevation value, and the coordinates in the horizontal direction, that is, three-dimensional data are obtained.

3 is a schematic diagram showing an example of a ground state in the real world. 3 illustrates a cross section of a portion of the real world, where features exist on a undulating terrain.

FIG. 4 is a schematic diagram showing DSM data generated by stereo matching processing from an image of a part of the real world shown in FIG. 3. Since the DSM data represents the height data of the outermost surface, for the ground surface obscured by the roof or the like, the DSM data indicates the height of the roof including the elevation value.

The reference parallax setting part 12 of FIG. 1 sets the parallax which gives the parallax of the reference | standard which searches the corresponding point in two images. For example, the reference parallax setting unit 12 sets a reference to a parallax that provides a height of a reference of a range to be searched for in stereo matching in real space. The search range setting unit 13 sets a range in which the stereo matching unit 11 searches for a corresponding point based on the point of the image giving the parallax of the reference set by the reference parallax setting unit 12. Usually, the predetermined range smaller than the range of an image is set as a search range of stereo matching.

4 conceptually illustrates a height corresponding to a reference elevation and a search range. For example, as a reference elevation, the surface indicated by the line SL of the predetermined height H from the height G of the origin of the altitude of the aircraft with respect to the inclined index as shown in FIG. The range between the line BL and the line UL is set to the height SH corresponding to the search range in the actual space, based on the plane of the line SL.

5 shows a search plane describing the search range. FIG. 5 shows the scanning line (epipolar line) A of the image converted from the aerial photo 101A and the scanning line (epipolar line) B corresponding to the scanning line A of the image converted from the aerial photo 101B. will be. The plane constituted by axes A and B is generally called the search plane. The center positions of the blocks on the axes A and B are represented by the vertical lines and the horizontal lines, respectively, and one correspondence between the two images is represented by the intersection of the vertical lines and the horizontal lines. The stereo matching unit 11 searches for the corresponding points of the two images on the scan line A and the scan line B. FIG. A 45 degree angle line in the search plane indicates that the parallax is constant, i.e., constant height, in the two images.

In FIG. 5, the line gl is a line of parallax corresponded to the height of the reference | standard of the ground, for example, and shall represent the origin of the height of an aircraft. Line sp is the height which gives a reference parallax, and corresponds to the line SL of FIG. The width SP corresponds to the height H in FIG. 4. The line u in FIG. 5 corresponds to the line UL in FIG. 4, and the line l corresponds to the line BL in FIG. 4. A range of widths R surrounded by lines u and l represents the search range.

The stereo matching unit 11 searches for a combination of correspondences between the lines A and B, for example, between the lines u and l in FIG. 5. The point of the scanning line B corresponding to the arbitrary point on the scanning line A is searched in the part between the line u of the vertical line passing through the point of the scanning line A, and the line l. When the point of the scanning line B is determined and the corresponding point of the scanning line A is searched, it is searched in the part between the line u of the horizontal line which passes through the point of the scanning line B, and the line l.

6 shows an example of a search range. If you know the elevation of the floor's ups and downs, you can determine the extent to which you search for features of the maximum height that is assumed. In FIG. 6, the parallax corresponding to ups and downs is shown by the line ls. The search range is represented by the lines u and l of the range R including the elevation difference of the relief on the basis of the parallax sp of the mean height of the relief plus the height of the assumed average of the features.

When the elevation of the index is known, the stereo matching process can be performed quickly by setting the search range including the height of the feature assumed to be at a high level. In addition, since the search range is limited, the possibility of misrecognizing a corresponding point of two images is reduced.

7 is a flowchart showing an example of the operation of the height facial expression processing according to the first embodiment. The image data input unit 10 inputs a plurality of images for stereo matching processing (step S11). The reference parallax setting unit 12 sets the standard parallax, for example, from the height of the feature assumed to be the elevation of the average of the region represented by the plurality of input images (step S12). The search range setting unit 13 sets the search range with respect to the standard parallax from the elevation difference of the area and the height of the feature (step S13).

The stereo matching unit 11 searches for the corresponding point of the other image in the set search range with respect to the point of one image, based on the point of the other image giving the parallax of the reference (step S14). The height calculation part 2 calculates the height corresponding to the point and the coordinate on the map of the point from the position in the image of each point associated with the some image (step S15).

According to the stereo image processing apparatus 1 of the first embodiment, stereo matching processing can be performed quickly by setting the parallax and the search range of the reference including the height of a feature assumed to be a high level difference. In addition, since the search range is limited, the possibility of misrecognizing a corresponding point of two images is reduced.

(Embodiment 2)

8 is a block diagram showing an example of the configuration of the stereo image processing apparatus 1 according to the second embodiment of the present invention. In Embodiment 2, the parallax and search range of a reference | standard are set based on the elevation data of map data.

The stereo image processing apparatus 1 of FIG. 8 includes a map data input unit 14 and an area divider 15 in addition to the configuration of the first embodiment. The map data input unit 14 inputs map data of a target region of the input image data. The map data includes elevation data at each point of the map mesh.

The area dividing unit 15 divides the input image data according to the elevation data. If the elevation difference of the elevation data is small, it may be set as one area without dividing the whole image. If the elevation difference exceeds a predetermined range in the map data of the image range, the image is divided into a plurality of areas.

The reference parallax setting unit 12 sets the parallax of the reference on the basis of the elevation data for each area divided by the area dividing unit 15. The search range setting unit 13 also sets the search range for each divided area in consideration of the height of the feature assumed by the elevation difference of the elevation data.

The stereo matching unit 11 searches for corresponding points of the plurality of images in accordance with the parallax and the search range of the reference set for each divided region, and extracts a pair of the corresponding points. The height calculation part 2 produces | generates DSM data by the principle of triangulation from the parallax of the corresponding point obtained by the stereo matching process.

9 shows an example of a search plane for setting a parallax and a search range of a reference for each divided region. In the example of FIG. 9, it is divided | segmented into four area | regions according to the relief ls of an indicator. The parallax of a reference | standard is made into sp1, sp2, sp3, and sp4 in each area | region. In the first area, the range of the width R1 bounded by u1 and l1 is set as the search range. Hereinafter, the range of the width R2 bounded by u2 and l2, the range of the width R3 bounded by u3 and l3, and the range of the width R4 bounded by u4 and l4 are defined as search ranges, respectively.

As shown in FIG. 9, the search range is smaller than that in FIG. 6. When there is a high or low level of relief, the search range can be limited to the range where the corresponding point exists by dividing the image range into a plurality of regions and setting the search range for each divided region. As a result, the stereo matching process can be performed quickly and accurately.

In addition, the area division can be set so that the elevation difference in the area is equal to or less than a predetermined value. Alternatively, the number of divisions may be determined according to the height difference of the image range.

10 is a flowchart showing an example of the operation of the height facial expression processing according to the second embodiment. When the image data input unit 10 inputs a plurality of images subjected to stereo matching processing (step S21), the map data input unit 14 inputs map data of the target area (step S22). The area dividing unit 15 divides the image into regions based on elevation data included in the map data (step S23).

The reference parallax setting unit 12 sets the standard parallax for each divided area based on the elevation data and the divided area (step S24). The search range setting unit 13 sets the search range for each region according to the height difference of the divided regions (step S25).

The stereo matching unit 11 searches for the corresponding point of the other image in the search range set for each region with respect to the point of one image, based on the point of the other image giving the parallax of the reference (step S26). The height calculation part 2 calculates the height corresponding to the point and the coordinate on the map of the point from the position in the image of each point associated with the some image (step S27).

According to the stereo image processing apparatus 1 of the second embodiment, the search range can be limited to the range where the corresponding point exists in accordance with the elevation data of the map data. As a result, the stereo image processing apparatus 1 can perform stereo matching processing quickly and accurately.

(Embodiment 3)

11 is a block diagram showing a configuration example of the stereo image processing apparatus 1 according to the third embodiment. The stereo image processing apparatus 1 of the third embodiment includes a reference point setting unit 16 in addition to the configuration of the first embodiment.

The reference point setting unit 16 selects a reference point for setting the parallax of the reference by a predetermined method from among the images input by the image data input unit 10. For example, the reference point setting unit 16 randomly selects a number of points according to the size and scale of the image as the reference point. Alternatively, the reference point setting unit 16 may select a point of a predetermined distribution.

The reference point setting unit 16 causes the stereo matching unit 11 to extract a pair of points corresponding to each other among the plurality of images for each of the points selected by the reference point setting unit 16. The reference point setting unit 16 calculates three-dimensional data of each selected point and sends the three-dimensional data of the selected point to the reference parallax setting unit 12.

The reference parallax setting unit 12 sets the standard parallax from a three-dimensional data received from the reference point setting unit 16 by a predetermined method. For example, the reference parallax setting unit 12 uses the average or median value of the height of the three-dimensional data as the reference elevation, and sets the parallax corresponding to the reference elevation as the reference parallax. Alternatively, the reference parallax setting unit 12 may set the standard parallax for each region by averaging for each predetermined divided region.

The search range setting unit 13 sets the search range in accordance with the parallax of the reference. For example, the search range setting unit 13 may set the search range in consideration of the dispersion of the height of the 3D data of the selected point. In addition, when the range of an image is divided into a plurality of areas, a search range may be set for each divided area.

The stereo matching unit 11 searches for corresponding points of the plurality of images in accordance with the set parallax search range of the set reference, and extracts a pair of corresponding points. If the parallax of the reference and the search range are set for each divided area, the corresponding point is searched in the search range. The height calculation part 2 produces | generates DSM data by the principle of triangulation from the parallax of the corresponding point obtained by the stereo matching process.

12 is a flowchart showing an example of the operation of the height facial expression processing according to the third embodiment. When the image data input unit 10 inputs a plurality of images for stereo matching processing (step S31), the reference point setting unit 16 selects a reference point in the image by a predetermined method (step S32). The stereo matching unit 11 performs a stereo matching process on the selected reference point, and extracts a corresponding pair from the plurality of images (step S33).

The reference point setting unit 16 calculates the height of the reference point from the corresponding pair of the selected points (step S34). The reference parallax setting part 12 sets the parallax of a reference based on the height of a reference point (step S35). Here, the reference parallax setting unit 12 may set the parallax of the reference based on the corresponding pairs of the selected points (that is, the reference point setting unit 16 does not have to calculate the height of the reference point). The search range setting unit 13 sets the search range in accordance with the parallax of the reference (step S36).

The stereo matching unit 11 searches for the corresponding point of the other image in the search range set for each region with respect to the point of one image, based on the point of the other image giving the parallax of the reference (step S37). The height calculation part 2 calculates the height corresponding to the point and the coordinate on the map of the point from the position in the image of each point associated with the some image (step S38).

According to the stereo image processing apparatus 1 of the third embodiment, even when there is no map data, the search range can be limited to a range in which corresponding points exist so as to be suitable for the image. As a result, the stereo image processing apparatus 1 can perform stereo matching processing quickly and accurately.

(Fourth Embodiment)

FIG. 13 is a block diagram showing a configuration example of the stereo image processing apparatus 1 according to the fourth embodiment. The stereo image processing apparatus 1 of Embodiment 4 inputs the reference point of Embodiment 3 from the outside. The stereo image processing apparatus 1 of FIG. 13 includes a reference point input unit 17 in addition to the configuration of the third embodiment.

The reference point input unit 17 inputs data indicating a position in the image of the reference point. For example, the reference point input unit 17 may display image data on a display device (not shown) and input a point selected on the screen. In addition, the reference point input unit 17 may input coordinates in the image as data.

The operations of the reference point setting unit 16, the reference parallax setting unit 12, the search range setting unit 13, and the stereo matching unit 11 are similar to those in the third embodiment.

In Embodiment 4, the reference point considered appropriate from the input image can be set. For example, it is considered appropriate for the elevation of the reference, such as an aerial beacon such as a high-rise building, a feature point of an iron tower, or a feature of an artificial feature such as a building, and a point capable of performing stereo matching processing accurately can be selected.

14 is a flowchart showing an example of the operation of the height facial expression processing according to the fourth embodiment. When the image data input unit 10 inputs a plurality of images for stereo matching processing (step S41), the reference point input unit 17 inputs data indicating the position of the reference point in the image (step S42). There may be more than one reference point to select.

The reference point setting unit 16 sends the input position of the input point to the stereo matching unit 11, and the stereo matching unit 11 performs stereo matching on the selected reference point, and sets the corresponding points in the plurality of images. Is extracted (step S43). The reference point setting unit 16 calculates the height of the reference point from the corresponding pair of the selected points (step S44). Subsequently, the corresponding height calculation (step S48) from the standard parallax setting (step S45) is the same as that of steps S35 to S38 in FIG.

According to the stereo image processing apparatus 1 of the fourth embodiment, in addition to the third embodiment, it is possible to select a point capable of accurately performing stereo matching processing with a parallax of an appropriate reference according to the image. As a result, the stereo image processing apparatus 1 can perform stereo matching processing quickly and accurately.

(Embodiment 5)

FIG. 15 is a block diagram showing a configuration example of the stereo image processing apparatus 1 according to the fifth embodiment. The stereo image processing apparatus 1 of Embodiment 5 inputs the reference point of Embodiment 3 and the pair corresponding to it in stereo matching from the exterior. The stereo image processing apparatus 1 of FIG. 15 includes a reference correspondence point input unit 18 instead of the reference point input unit 17 and the reference point setting unit 16 of the third embodiment.

The reference correspondence point input unit 18 inputs data indicating the position of the reference point in the image and the position of the point in the other image corresponding to stereo matching. For example, the reference correspondence point input unit 18 may display two image data on a display device (not shown) and input a pair of correspondence points selected on the screen. In addition, the reference | correspondence point input part 18 may input the coordinate of the pair of the correspondence point in two images as data.

The operations of the reference parallax setting unit 12, the search range setting unit 13, and the stereo matching unit 11 are similar to those in the first embodiment.

In Embodiment 5, the pair of reference | correspondence points considered appropriate from the input image can be set. For example, a corresponding point that is considered to be appropriate for the elevation of the reference, such as an aerial sign such as a high-rise building, a feature point of a steel tower, or a feature of an artificial feature such as a building, can be selected.

16 is a flowchart illustrating an example of the operation of the height facial expression processing according to the fifth embodiment. When the image data input unit 10 inputs a plurality of images subjected to stereo matching processing (step S51), the reference correspondence point input unit 18 indicates the position of the pair of reference correspondence points corresponding to each other by stereo matching in two images among those images. Data is input (step S52). There may be a plurality of reference points to select.

Subsequently, the corresponding height calculation (step S56) from the standard parallax setting (step S53) is the same as that of steps S35 to S38 in FIG.

According to the stereo image processing apparatus 1 of the fifth embodiment, the parallax of the reference and the search range can be set by selecting the corresponding points that are stereo matched with the appropriate parallax according to the image. As a result, the stereo image processing apparatus 1 can perform stereo matching processing quickly and accurately.

17 is a block diagram showing an example of a physical configuration when the stereo image processing apparatus 1 is mounted on a computer. The stereo image processing apparatus 1 according to the present embodiment can be realized by a hardware configuration similar to that of a general computer apparatus. As shown in FIG. 17, the stereo image processing apparatus 1 includes a control unit 21, a main memory unit 22, an external storage unit 23, an operation unit 24, a display unit 25, and an input / output unit 26. It is provided. The main memory unit 22, the external storage unit 23, the operation unit 24, the display unit 25, and the input / output unit 26 are all connected to the control unit 21 via the internal bus 20.

The control unit 21 is composed of a CPU (Central Processing Unit) or the like, and performs stereo matching processing in accordance with the control program 30 stored in the external storage unit 23.

The main memory section 22 is composed of a RAM (Random-Access Memory) or the like. The main memory section 22 loads the control program 30 stored in the external storage section 23 and is used as a work area of the control section 21.

The external storage unit 23 is composed of a nonvolatile memory such as a flash memory, a hard disk, a digital versatile disc random-access memory (DVD-RAM), a digital versatile disc re-writable (DVD-RW), and controls the above process. The control program 30 for causing (21) to be stored is stored in advance. Moreover, according to the instruction | indication of the control part 21, the data which this control program 30 stores is supplied to the control part 21, and the data supplied from the control part 21 is stored.

The operation unit 24 is composed of a pointing device such as a keyboard and a mouse and the like, and an interface device for connecting the keyboard and pointing device to the internal bus 20. Via the operation unit 24, an instruction such as input of image data, transmission and reception, designation of an image to be displayed, a position in an image of a reference point for setting a reference elevation, and the like are input to the control unit 21.

The display unit 25 is composed of a cathode ray tube (CRT), a liquid crystal display (LCD), or the like, and displays an image or a result of stereo matching.

The input / output unit 26 is composed of a wireless transceiver, a wireless modem or a network termination device, and a serial interface or a local area network (LAN) interface connected thereto. Via the input / output unit 26, image data can be received and the result of the calculation can be transmitted.

Image data input unit 10, stereo matching unit 11, reference parallax setting unit 12, search range setting of the stereo image processing apparatus 1 shown in FIG. 1, 8, 11, 13, or 15 The control program 30 controls the processing of the unit 13, the map data input unit 14, the area divider 15, the reference point setting unit 16, the reference point input unit 17, and the reference correspondence point input unit 18. (21), the main memory unit 22, the external storage unit 23, the operation unit 24, the display unit 25 and the input / output unit 26 and the like are processed by using as resources.

In addition, the following structures are included as a preferable modification of this invention.

With respect to the stereo matching processing device according to the first aspect of the present invention,

Preferably, the reference parallax setting unit divides each of the plurality of images into two or more regions respectively corresponding to two or more regions divided outside the plurality of images, and sets the parallax of the reference in each region. The search range setting unit sets a predetermined range smaller than the range of the image as the search range for stereo matching based on the point of the image giving the parallax of the reference set in the respective areas.

Preferably, the stereo matching processing apparatus includes a map data acquisition unit for acquiring elevation data of a map corresponding to the plurality of images, wherein the reference parallax setting unit is based on the elevation data acquired by the map data acquisition unit. Set the time difference of.

Alternatively, the reference parallax setting unit may calculate a parallax by stereo matching on a point selected from the plurality of images by a predetermined method, and set the parallax of the reference based on the calculated parallax.

The stereo matching processing apparatus further includes a reference point input unit for acquiring a position in the image of the point for which the parallax is obtained from among the plurality of images, and the reference parallax setting unit performs parallax by stereo matching with respect to a point acquired by the reference point input unit. May be calculated and the parallax of the reference may be set based on the calculated parallax.

Alternatively, the stereo matching processing apparatus includes a correspondence point input unit for inputting a pair of points corresponding to stereo matching from the plurality of images, and the reference parallax setting unit is applied to a pair of corresponding points input by the correspondence point input unit. You may set the parallax of the said reference | standard based on the parallax which becomes.

With respect to the stereo matching processing method according to the second aspect of the present invention,

Preferably, the reference parallax setting step divides one set of images to be subjected to the stereo matching process into two or more regions respectively corresponding to two or more regions divided outside the image, and sets the parallax of the reference in each region. The step of setting the search range is characterized by setting a predetermined range smaller than the range of the image as the search range for stereo matching, based on the point of the image giving the parallax of the reference set in the respective areas. .

Preferably, the stereo matching processing method includes a map data acquisition step of acquiring elevation data of a map corresponding to the plurality of images, wherein the reference parallax setting step is based on elevation data acquired in the map data acquisition step. The parallax of the reference is set.

Alternatively, the reference parallax setting step may calculate a parallax by stereo matching on a point selected from the plurality of images by a predetermined method, and set the parallax of the reference based on the calculated parallax.

A stereo matching processing method further includes a reference point input step of acquiring a position in an image of a point for which parallax is to be obtained among the plurality of images, wherein the reference parallax setting step is adapted to stereo matching with respect to a point acquired in the reference point input step. By calculating the parallax, and on the basis of the calculated parallax.

Alternatively, the stereo matching processing method includes a correspondence point input step of inputting a pair of points corresponding to stereo matching from the plurality of images, and the reference parallax setting step includes a corresponding point input in the correspondence point input step. You may set the parallax of the said reference | standard based on the parallax provided in a tank.

In addition, the above-mentioned hardware configuration and flowchart are an example, and can be changed and modified arbitrarily.

Center for performing processing of the stereo image processing apparatus 1 composed of the control unit 21, the main memory unit 22, the external storage unit 23, the operation unit 24, the input / output unit 26, the internal bus 20, and the like. The part to be realized can be realized using a normal computer system, regardless of a dedicated system. For example, by storing and distributing a computer program for performing the above operation on a computer-readable recording medium (flexible disc, CD-ROM, DVD-ROM, etc.), and installing the computer program on the computer, The stereo image processing apparatus 1 which performs the process of may be comprised. In addition, the stereo image processing apparatus 1 may be configured by storing the computer program in a storage device of a server device on a communication network such as the Internet and downloading it by a normal computer system.

In addition, when the function of the stereo image processing apparatus 1 is realized by sharing the OS (operating system) and the application program or by cooperating with the OS and the application program, only the application program portion is stored in the recording medium or the storage device. You may save it.

It is also possible to superimpose a computer program on a carrier wave and distribute it through a communication network. For example, the computer program may be posted on a bulletin board system (BBS) on a communication network, and the computer program may be distributed through a network. Then, the computer program may be started, and under the control of the OS, the computer program may be executed in the same manner as in other application programs to execute the above process.

In addition, about this application, the priority based on Japanese Patent Application No. 2008-300221 is claimed, and the specification, a claim, and the drawing of Japanese Patent Application No. 2008-300221 are referred to in this specification as a reference.

1: Stereo Image Processing Unit
2: height calculation unit
10: image data input unit
11: stereo matching unit
12: reference parallax setting unit
13: search range setting unit
14: map data input unit
15: region divider
16: reference point setting unit
17: reference point input unit
18: reference correspondence point input unit
21:
22: main memory
23: external storage
24: control panel
25:
26: input and output unit
30: control program

Claims (12)

A stereo matching processing device,
An image data acquisition unit for acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;
A reference parallax setting unit for dividing the plurality of images into two or more corresponding regions, respectively, and setting a standard parallax in each region;
A search range setting unit for setting a predetermined range smaller than the range of the image as a search range for stereo matching on the basis of the point of the image giving the parallax of the reference in each area set by the reference parallax setting unit;
With respect to an arbitrary point in one image of the plurality of images, the other image in the search range set by the search range setting unit is based on the point of another image which gives the parallax of the reference set by the reference parallax setting unit. A search unit for searching for a corresponding point;
Map data acquisition unit for obtaining elevation data of the map corresponding to the plurality of images.
Including,
The reference parallax setting unit divides the image data acquired by the image data acquisition unit into a plurality of areas according to the elevation data acquired by the map data acquisition unit, and sets the parallax of the reference based on the elevation data for each divided area. Stereo matching processing apparatus, characterized in that the setting. A stereo matching processing device,
An image data acquisition unit for acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;
A reference parallax setting unit for dividing the plurality of images into two or more corresponding regions, respectively, and setting a standard parallax in each region;
A search range setting unit for setting a predetermined range smaller than the range of the image as a search range for stereo matching on the basis of the point of the image giving the parallax of the reference in each area set by the reference parallax setting unit;
With respect to an arbitrary point in one image of the plurality of images, the other image in the search range set by the search range setting unit is based on the point of another image which gives the parallax of the reference set by the reference parallax setting unit. Search unit for searching the corresponding point
Including,
The reference parallax setting unit calculates parallaxes by stereo matching on points selected from the plurality of images by a predetermined method, and sets the parallaxes of the reference based on the calculated parallaxes. Device. As a stereo matching processing method,
An image data acquisition step of acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;
A reference parallax setting step of dividing the plurality of images into two or more corresponding regions, respectively, and setting a standard parallax in each region;
A search range setting step of setting, as a search range for stereo matching, a predetermined range smaller than the range of the image on the basis of the point of the image giving the parallax of the reference in each area set in the reference parallax setting step;
With respect to any point in one image of the plurality of images, the other image in the search range set in the search range setting step is based on the point of another image which gives the parallax of the reference set in the reference parallax setting step. Search step to search for corresponding point
Including,
In the reference parallax setting step, the parallax is calculated by stereo matching with respect to a point selected from the plurality of images by a predetermined method, and the parallax of the reference is set based on the calculated parallax. Treatment method. A computer-readable recording medium storing a program for controlling a computer, the computer comprising:
An image data acquisition unit for acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;
A reference parallax setting unit for dividing the plurality of images into two or more corresponding regions, respectively, and setting a standard parallax in each region;
A search range setting unit for setting a predetermined range smaller than the range of the image as a search range for stereo matching on the basis of the point of the image giving the parallax of the reference in each area set by the reference parallax setting unit;
With respect to an arbitrary point in one image of the plurality of images, the other image in the search range set by the search range setting unit is based on the point of another image which gives the parallax of the reference set by the reference parallax setting unit. Function as a search unit for searching for a corresponding point,
And the reference parallax setting unit functions to calculate a parallax by stereo matching on a point selected from the plurality of images by a predetermined method, and to set the parallax of the reference based on the calculated parallax. Readable recording medium. The method of claim 1,
The reference parallax setting unit calculates parallaxes by stereo matching on points selected from the plurality of images by a predetermined method, and sets the parallaxes of the reference based on the calculated parallaxes. Processing unit. The method of claim 2,
A reference point input unit for acquiring a position in an image of a point for calculating a parallax among the plurality of images,
And the reference parallax setting unit calculates parallaxes by stereo matching on points acquired by the reference point input unit, and sets the parallaxes of the reference based on the calculated parallaxes. The method of claim 2,
A corresponding point input unit for inputting a pair of points corresponding to stereo matching from the plurality of images,
And the reference parallax setting unit sets the parallax of the reference based on a parallax given by a pair of corresponding points input by the corresponding point input unit. As a stereo matching processing method,
An image data acquisition step of acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;
A reference parallax setting step of dividing the plurality of images into two or more corresponding regions, respectively, and setting a standard parallax in each region;
A search range setting step of setting, as a search range for stereo matching, a predetermined range smaller than the range of the image data on the basis of the point of the image giving the parallax of the reference in each area set in the reference parallax setting step;
With respect to any point in one image of the plurality of images, the other image in the search range set in the search range setting step is based on the point of another image which gives the parallax of the reference set in the reference parallax setting step. A search step of searching for a corresponding point,
Map data acquisition step of acquiring elevation data of a map corresponding to the plurality of images.
Including,
The reference parallax setting step divides the image data acquired in the image data acquisition step into a plurality of areas according to the elevation data acquired in the map data acquisition step, and for each divided area, the parallax of the reference based on the elevation data. Stereo matching processing method characterized in that the setting. 9. The method of claim 8,
In the reference parallax setting step, a parallax is calculated by stereo matching with respect to a point selected from the plurality of images by a predetermined method, and the parallax of the reference is set based on the calculated parallax. Match processing method. The method of claim 3,
A reference point input step of acquiring a position in an image of a point for calculating parallax among the plurality of images,
The reference disparity setting step includes: calculating a disparity by stereo matching with respect to a point acquired in the reference point input step, and setting the disparity of the reference based on the calculated disparity. The method of claim 3,
A correspondence point input step of inputting a pair of correspondence points corresponding to each other for stereo matching from the plurality of images,
And the step of setting the reference parallax sets the parallax of the reference based on the parallax given by the pair of corresponding points input in the corresponding point input step. A computer-readable recording medium storing a program for controlling a computer, the computer comprising:
An image data acquisition unit for acquiring image data of a plurality of images photographing a predetermined area from a plurality of different positions;
A reference parallax setting unit for dividing the plurality of images into two or more corresponding regions, respectively, and setting a standard parallax in each region;
A search range setting unit for setting a predetermined range smaller than the range of the image as a search range for stereo matching on the basis of the point of the image giving the parallax of the reference in each area set by the reference parallax setting unit;
With respect to an arbitrary point in one image of the plurality of images, the other image in the search range set by the search range setting unit is based on the point of another image which gives the parallax of the reference set by the reference parallax setting unit. A search unit for searching for a corresponding point;
Function as a map data acquisition unit for acquiring elevation data of a map corresponding to the plurality of images,
The reference parallax setting unit divides the image data acquired by the image data acquisition unit into a plurality of areas according to the elevation data acquired by the map data acquisition unit, and sets the parallax of the reference based on the elevation data for each divided area. And a computer readable recording medium.

Images