KR101260213B1 - Image processing device, image processing method, and recording medium - Google Patents

Abstract

Based on the block images 61a (m, n) and 62a (m, n) corresponding to each other, matching images SMA1 and SMA2 corresponding to each other are set, respectively. When the corresponding point of the feature points included in one matching image cannot be extracted from the other matching image, block images adjacent to the generation direction of parallax are combined to form a new block image 61a (m, ( n-1)-(n + 1)) and 62a (m, (n-1)-(n + 1))), respectively. Next, based on the newly defined block image, the matching image is set again.

Description

Image processing apparatus, image processing method and recording medium {IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, AND RECORDING MEDIUM}

The present invention relates to a computer-readable recording medium recording an image processing apparatus, an image processing method and a program, and more particularly, an image processing apparatus which performs stereo matching processing on a set of images photographed from different positions; An image processing method and a recording medium for performing stereo matching processing on a set of images photographed from different positions.

In the field of terrain analysis and the like, a method of calculating three-dimensional information such as the shape of a land, the position and the height of a building is frequently used by performing stereo matching on a set of images obtained by photographing from different viewpoints (Example See, for example, Patent Document 1). The stereo matching process corresponds to a feature point in one of the two images, for example, a corner of a building taken in the image or a portion rapidly protruding from the surface by using an area correlation method or the like. And a corresponding point in the other image and extracting three-dimensional information including the positional information and the height information of the subject based on the extracted feature point and the positional information of the corresponding point.

In this stereo matching process, when the image to be processed is, for example, an image obtained by photographing an urban part in which a large number of high-rise buildings and the like exist, the number of feature points in one image is enormous. Therefore, in order to shorten the time required for processing, a method of dividing each set of images to be processed into a plurality of images and performing stereo matching processing for each divided image (hereinafter, simply referred to as a divided image) is proposed. (For example, refer nonpatent literature 1).

Patent Document 1: Japanese Patent Application Laid-open No. Hei 8-16930

Non-Patent Document 1: Proceedings of the Korea Information Processing Society Conference, VOL.64th, NO.4 (PAGE. 4.767-4.770)

In the above-described method, since the amount of data handled at a time in the stereo matching process is reduced, it is possible to perform a process for a set of images in a short time. However, in the upper part of a high building etc., such as a high-rise building, the parallax between a set of images used as a process object becomes large, and the correspondence point corresponding to a characteristic point may not be included in the division | segmentation image which corresponds. In this case, stereo matching processing cannot be performed or the processing results are insufficient.

SUMMARY OF THE INVENTION The present invention has been made under the foregoing circumstances, and an object thereof is to realize an improvement in processing accuracy while increasing the speed of image processing.

An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that performs stereo matching processing on a first image and a second image photographed from different positions, each of the first image and the second image. A setting unit for dividing into a plurality of images, setting a divided image of the first image, a divided image of a second image corresponding to the divided image, as a first matching image and a second matching image, respectively, and the first matching An extracting unit for extracting a corresponding point included in the second matching image corresponding to a point included in the image, a split image set as the first matching image when the extracting unit cannot extract the corresponding point, and the A combined image obtained by combining another divided image adjacent to the divided image with each of the divided images set as the second matching image, is a first matching image and a second matching image, respectively. Characterized in that it comprises a setting and reset state, the operation for calculating the point, and three-dimensional information of the corresponding point is included in each of the matching image of the second image included in the image matching of the first image.

Moreover, the image processing apparatus which concerns on the 2nd viewpoint of this invention is an image processing apparatus which performs a stereo matching process with respect to the 1st image and the 2nd image photographed from a different position, The said 1st image and the said 2nd image Each is divided into a plurality of images, and the first matching image and the first matching image are divided into a divided image of the first image and a margin area around it, and a divided image of the second image corresponding to the divided image and a margin area around the first image, respectively. A setting unit for setting a matching image, an extraction unit for extracting a corresponding point included in the second matching image corresponding to a point included in the first matching image, and the extraction unit cannot extract the corresponding point Is obtained by combining another divided image adjacent to the divided image with each of the divided images included in the first matching image and the divided images included in the second matching image. A reset unit for setting the sum image and the margin area around the sum image to the first matching image and the second matching image, a point included in the first matching image, and three corresponding points included in the second matching image. And an operation unit for calculating the dimensional information.

Further, an image processing method according to a third aspect of the present invention is an image processing method for performing stereo matching processing on a first image and a second image photographed from different positions, wherein the first image and the first image are performed. A setting step of dividing each of the two images into a plurality of images and setting the divided image of the first image and the divided image of the second image corresponding to the divided image as a first matching image and a second matching image, respectively; An extraction step of extracting a corresponding point included in the second matching image corresponding to a point included in the first matching image; and when the corresponding point cannot be extracted in the extraction step, the first matching image is set to the first matching image. The first matching image is obtained by combining the divided image and the combined image obtained by combining the divided image set as the second matching image with another divided image adjacent to the divided image, respectively. And a resetting step of resetting to a second matching image, and calculating a three-dimensional information of each of the points included in the matching image of the first image and the corresponding points included in the matching image of the second image. It is characterized by.

Further, an image processing method according to a fourth aspect of the present invention is an image processing method for performing stereo matching processing on a first image and a second image photographed from different positions, wherein the first image and the first image are performed. Each of the two images is divided into a plurality of images, and the first matching image is divided into a divided image of the first image and a margin area around the first image, and a divided image of the second image corresponding to the divided image and a margin area around the first image, respectively. And a setting step of setting a second matching image, an extraction step of extracting a corresponding point included in the second matching image corresponding to a point included in the first matching image, and extracting the corresponding point in the extraction step. In the case where it was not possible, each of the divided images included in the first matching image and the divided images included in the second matching image are combined with other divided images adjacent to the divided images. And a resetting step of setting the combined image and the margin area around the same as the first matching image and the second matching image, the points included in the first matching image, and the corresponding points included in the second matching image, respectively. Comprising a calculation process for calculating the three-dimensional information of the.

The computer-readable recording medium having recorded thereon the program according to the fifth aspect of the present invention divides a computer into a plurality of images by dividing each of the first image and the second image photographed from different positions. A setting unit for setting a divided image of the image and a divided image of the second image corresponding to the divided image as a first matching image and a second matching image, respectively, and corresponding points included in the first matching image. An extraction unit for extracting a corresponding point included in a second matching image, and a split image set as the first matching image, and a divided image set as the second matching image, respectively, when the extraction unit cannot extract the corresponding point. A resetting unit for resetting the combined image obtained by combining other divided images adjacent to the divided images to the first matching image and the second matching image, respectively; And a program for functioning as an operation unit for calculating three-dimensional information of each point included in the matching image of the first image and the corresponding point included in the matching image of the second image.

A computer-readable recording medium having recorded thereon a program according to the sixth aspect of the present invention divides a computer into a plurality of images by dividing each of a first image and a second image photographed from different positions. A setting unit for setting the divided image and the margin region around it, the divided image of the second image corresponding to the divided image, and the margin region around the divided image as a first matching image and a second matching image, respectively, and the first matching An extracting unit for extracting a corresponding point included in the second matching image corresponding to a point included in the image, a split image included in the first matching image when the extracting unit could not extract the corresponding point, and The combined image obtained by combining another divided image adjacent to the divided image to each of the divided images included in the second matching image, and the margin area around the first sheet again. A resetting unit for setting an image and a second matching image, a program for functioning as a calculating unit for calculating three-dimensional information of each of the points included in the first matching image and the corresponding points included in the second matching image; It is characterized by.

Stereo matching processing for different sets of images can be performed in a short time and with high accuracy.

1 is a block diagram of a stereo image processing apparatus according to an embodiment of the present invention.
2 is a diagram for explaining image data.
3A is a diagram showing an image as image data;
3B is a diagram showing an image as image data;
4A is a diagram showing a matching image based on the divided image.
4B is a diagram showing a matching image based on the divided image.
5A is a diagram illustrating a matching image based on the divided image.
5B is a diagram showing a matching image based on the divided image.
6A illustrates a matching image based on the combined image.
6B is a diagram showing a matching image based on the combined image.
7 is a flowchart showing the operation of the stereo image processing apparatus.
8A is a diagram for explaining a modification of stereo image processing.
8B is a diagram for explaining a modification of stereo image processing.
Fig. 9 is a block diagram showing a physical configuration example in the case where a stereo image processing apparatus is mounted on a computer.

EMBODIMENT OF THE INVENTION Hereinafter, one Embodiment of this invention is described based on FIGS. 1 is a block diagram of a stereo image processing apparatus 10 according to the present embodiment. As shown in FIG. 1, the stereo image processing apparatus 10 includes a data input unit 11, an image extraction unit 12, an image division unit 13, a matching image setting unit 14, and a corresponding point extraction unit 15. ), A matching miss detector 16, a split image combiner 17, and a three-dimensional information calculator 18.

Image data is input to the data input unit 11 from an external device such as an imaging device. This image data is a picked-up image etc. which were obtained by imaging an index | index by an imaging device etc., for example. 2 is a diagram for explaining image data. In this embodiment, as shown in FIG. 2 as an example, two images acquired by imaging the area | region on the index F containing the building 71 and the building 72 are input while moving a camera to an X-axis direction. The explanation will be made as follows. In addition, for convenience of description, the optical axis of the digital camera 70 in the position P1 shown by the broken line of FIG. 2, and the optical axis of the digital camera 70 in the position P2 shown by the solid line shall be parallel, It is assumed that epipolar lines coincide between images. In addition, an epipolar line means the straight line which can be drawn as an existence range of the point corresponding to the point in the other image with respect to the arbitrary point in one image in a stereo image.

The image extraction unit 12 detects an overlapped region from each of a set of images, and extracts an image corresponding to this region from each of a set of images. 3A and 3B are diagrams showing an image as image data. As an example, FIG. 3A shows the image 61 which the digital camera 70 picked up from the position P1, and FIG. 3B shows that the digital camera 70 picked up from the position P2 on the + X side of the position P1. Image 62 is shown. The image extraction part 12 compares the image 61 and the image 62, and extracts the extraction image 61a, 62a which the area | region common to each other is imprinted from each of the image 61 and the image 62. FIG. .

The image dividing unit 13 is, for example, a block in which each of the extracted image 61a extracted from the image 61 and the extracted image 62a extracted from the image 62 is arranged in a matrix form of three rows and five columns. Split into images. Hereinafter, the n-th block image of the m-th column of the extracted image 61a is represented by 61a (m, n), and the n-th block image of the m-th column of the extracted image 62a is referred to 62a (m, n).

The matching image setting unit 14 corresponds to each other based on the block image 61a (m, n) of the extracted image 61a and the block image 62a (m, n) of the extracted image 62a. Set a matching image. 4A and 4B are diagrams showing a matching image based on the divided image. For example, when setting a matching image based on the block image 61a (1, 1) and the block image 62a (1, 1), as will be understood when referring to Figs. 4A and 4B. The matching image setting unit 14 adds a margin area M around the block image 61a (1, 1) and the block image 62a (1, 1). And the matching image setting part 14 sets the area | region containing the block image 61a (1, 1) and the margin area M to the matching image SMA1 (1, 1) to which a stereo matching process is performed, and blocks The area including the image 62a (1, 1) and the margin area M is set as the matching image SMA2 (1, 1). Hereinafter, the matching image setting unit 14 includes block images 61a (1, 2) to block images 61a (3 and 5), and block images 62a (1 and 2) to block images 62a ( 3, 5)), the same process is performed.

In addition, this margin area M is set so that when there is a parallax between the feature point and the corresponding point, the corresponding point corresponding to this feature point is included in the matching image corresponding to the matching image including the feature point. For this reason, the larger the margin area, the larger the parallax feature point and the corresponding point are included in the corresponding matching image.

The correspondence point extraction unit 15 extracts a correspondence point included in the matching image SMA2 (m, n) with respect to the feature points included in the matching image SMA1 (m, n) corresponding to each other. This process is, for example, an area correlation method by examining a correlation between a small area in an image included in matching image SMA1 (m, n) and a small area in an image included in matching image SMA2 (m, n). Or the like.

For example, as will be seen when referring to Figs. 4A and 4B, the correspondence point extraction unit 15 corresponds to the feature points a1 to a4 of the building 71 included in the matching image SMA1 (1, 1). As a point, points b1 to b4 included in the matching image SMA2 (1, 1) are extracted.

The matching miss detection unit 16 determines whether all of the corresponding points corresponding to the feature points included in the matching image SMA1 (m, n) exist in the matching image SMA2 (m, n).

For example, as can be seen by referring to Figs. 4A and 4B, the matching miss detection unit 16 has all the corresponding points b1 to b4 with respect to the feature points a1 to a4 included in the matching image SMA1 (1, 1), When it is contained in matching image SMA2 (1, 1), it is determined that extraction of a corresponding point was successful. 5A and 5B are diagrams showing a matching image based on the divided image. On the other hand, for example, referring to FIG. 5A and FIG. 5B, the matching miss detection unit 16 matches the matching images SMA1 (2, 3) with the feature points c1 and the feature points c1 to c4 of the building 72. When only the feature point c3 is included and the matching image SMA2 (2, 3) does not include the corresponding point d1 and the corresponding point d3 corresponding to the feature point c1 and the feature point c3 of the building 72, it is determined that extraction of the corresponding point has failed. .

The three-dimensional information calculation unit 18 calculates three-dimensional information of the feature points of the matching image SMA1 and the corresponding points corresponding to the feature points extracted from the matching image SMA2. Specifically, for example, by using the method used for triangulation and the position of the feature point and the corresponding point which make the viewpoint of the digital camera 70 the origin at the position P1, and the method of triangulation, Three-dimensional information (DSM (Digital Surface Map Data) data) including height and the like is generated.

When the match detection unit 16 detects a matching miss, the split image combining unit 17 generates a parallax between the image 61 and the image 62 in the block image included in the matching image in which the miss is detected. By combining adjacent block images in the X-axis direction, which is the direction, new block images of the image 61 and the image 62 are defined.

6A and 6B are diagrams showing a matching image based on the combined image. 6A and 6B, for example, as shown in FIG. 6A, the split image combiner 17 has a block image 61a (2, 3) in the block image 61a (2, 3). The block image 61a (2, 2) and the block image 61a (2, 4) adjacent to the + X side and the -X side of the image are combined to define a new block image 61a (2, 2-4). do. Further, the block image 62a (2, 2) and the block image 62a adjacent to the block image 62a (2, 3) on the + X side and -X side of the block image 62a (2, 3). (2, 4) are combined to define a new block image 62a (2, 2-4).

When the block image is defined by the split image combiner 17, the matching image setting unit 14 sets the matching image again based on the block image.

6A and 6B, for example, the matching image setting unit 14 includes a block image 61a (2, 2-4) and a block image 62a (2, 2-4). A margin area M is added around)). And the matching image setting part 14 turns the area | region containing the block image 61a (2, 2-4) and the margin area M into matching image SMA1 (2, 2-4) to which a stereo matching process is performed. The area including the block image 62a (2, 2-4) and the margin area M is set to the matching image SMA2 (2, 2-4) where stereo matching processing is performed.

If the matching image SMA1 (2, 2-4) and the matching image SMA2 (2, 2-4) are set, the correspondence point extracting unit 15 is provided to the feature points included in the matching image SMA1 (2, 2-4). The corresponding point included in the matching image SMA2 (2, 2-4) is extracted.

As can be seen by referring to Figs. 6A and 6B, the matching image SMA2 (2, 2) as a corresponding point to the feature points c1 to c4 of the building 72 included in the matching image SMA1 (2, 2-4) here. The points d1 to d4 included in 2-4) are extracted. In this case, since the corresponding points of the feature points c1 to c4 included in the matching image SMA1 (2, 2-4) are all included in the matching image SMA2 (2, 2-4), the matching miss detector 16 No matching miss is detected.

Next, the operation of the stereo image processing apparatus 10 will be described using the flowchart shown in FIG. 7. When the image data is input to the data input unit 11, the stereo image processing apparatus 10 starts a series of processes shown in the flowchart of FIG.

In the first step S101, the image extraction unit 12 corresponds to an image 61 input to the data input unit 11 shown in FIGS. 3A and 3B, and an image corresponding to an area overlapping each other from the image 62. (61a, 62a) are extracted. In addition, as long as an image is a common (overlapping) region, the image extraction unit 12 can extract an arbitrary region.

In following step S102, the image division part 13 divides the extracted image 61a, 62a shown to FIG. 3A and FIG. 3B into the block image 61a (m, n), 62a (m, n). do. The image dividing unit 13 can also divide an image into a plurality of regions having arbitrary shapes.

In the next step S103, the matching image setting unit 14 adds a margin area M to each of the block images 61a (m, n) and 62a (m, n), as shown in Figs. 4A and 4B. Then, matching images SMA1 and SMA2 are set. In addition, the margin area M is an arbitrary area corresponding to the outer edges of the block image 61a and the block image 62a.

In following step S104, the correspondence point extraction part 15 extracts the correspondence point contained in matching image SMA2 corresponding to the feature point contained in matching image SMA1. For example, as shown in FIGS. 4A and 4B, the corresponding point extracting unit 15 is a matching point corresponding to the feature points a1 to a4 of the building 71 included in the matching image SMA1 (1, 1). The points b1 to b4 included in the images SMA2 (1, 1) are extracted.

In following step S105, the matching miss detection part 16 detects a matching miss from the fact whether the corresponding point corresponding to the feature point contained in matching image SMA1 is contained in all the matching image SMA2. For example, as shown in FIGS. 5A and 5B, the matching miss detection unit 16 includes only the feature point c1 and the feature point c3 among the feature points c1 to c4 of the building 72 in the matching image SMA1 (2, 3). When the matching image SMA2 (2, 3) does not contain the corresponding point d1 and the corresponding point d3 corresponding to the feature point c1 and the feature point c3 of the building 72, it is determined that extraction of the correspondence point has failed.

When a matching miss is detected (step S105; YES), the process of step S106 is executed, and when a matching miss is not detected (step S105; NO), the process of step S107 is executed.

In step S106, the divided image combining unit 17 is a block image adjacent to the block image included in the matching image from which the miss is detected, in the X-axis direction, which is a direction in which parallax occurs in the image 61 and the image 62. By combining, a new block image is defined. For example, as shown in Figs. 6A and 6B, the divided image combiner 17 adds + X of the block image 61a (2, 3) to the block image 61a (2, 3). A new block image 61a (2, 2-4) is defined by combining the block image 61a (2, 2) and the block image 61a (2, 4) adjacent to the side and -X side. Further, the block image 62a (2, 2) and the block image 62a adjacent to the block image 62a (2, 3) on the + X side and -X side of the block image 62a (2, 3). (2, 4) are combined to define a new block image 62a (2, 2-4).

In step S107, the three-dimensional information calculation unit 18 generates three-dimensional information (DSM data) based on the positional information between the feature point of the matching image SMA1 and the corresponding point corresponding to the feature point extracted from SMA2.

As described above, in the present embodiment, the matching image SMA1 and the matching image SMA2 corresponding to each other are set based on the block image 61a (m, n) and the block image 62a (m, n). When the corresponding points of the feature points included in the matching image SMA1 cannot be extracted from the matching image SMA2, the block images 61a (m, n) adjacent to each other in the generation direction of the parallax are combined to form a new block image ( 61a (m, (n-1)-(n + 1)) and 62a (m, (n-1)-(n + 1)) are defined, and these block images 61a (m, (n-1) Based on)-(n + 1)) and 62a (m, (n-1)-(n + 1))), the matching picture SMA1 and the matching picture SMA2 are set again. As a result, the feature point and the corresponding point are included in the matching image corresponding to each other. Therefore, even when the stereo matching process for the screen 61 and the screen 62 is performed for each divided image, it can be performed with high precision.

In addition, in this embodiment, as will be understood when referring to FIG. 6A or FIG. 6B, the divided image combiner 17 combines three block images to form an image image, but is not limited thereto. When the parallax between the feature point in the image 61 and the corresponding point in the image 62 is large, four or more images may be combined and a matching image may be set based on the combined image.

8A and 8B are diagrams for explaining a modification of the stereo image processing. 8A and 8B, for example, in the image 61, the building is taken over two block images. In the image 62, the building is placed in one block image. In the case of being taken in the entered state, or the like, two block images may be combined to form a combined image.

In addition, in this embodiment, although each extraction image 61a, 62a was divided into 15 block images, it is not limited to this, Each of extraction image 61a, 62a may be further subdivided, and also less than 15 blocks You may divide into an image.

In addition, in this embodiment, for convenience of explanation, it demonstrated as the epipolar line coinciding between the image 61 and the image 62, However, the epipolar line in the image 61 and the epipolar in the image 62 are described. If the lines do not coincide with each other, the image 61 and the image 61 may be used, for example, using the method disclosed in Japanese Patent Application Laid-Open No. 2002-157576 before the processing by the image extraction unit 12 is executed. The parallelization process may be performed such that the corresponding point in the image 62 is on the same line (for example, on a line parallel to the X axis). By performing this process, the direction of generation of parallax between both images becomes the X-axis direction, and three-dimensional data can be similarly generated by performing the process described in the above embodiment.

In the present embodiment, the margin areas M around the block images 61a (m, n) and 62a (m, n) and the block images 61a (m, n) and 62a (m, n). Although the area including the above is set to the matching images SMA1 and SMA2, the present invention is not limited to this, and the matching images SMA1 and SMA2 having the same size as the divided images may be set without including the margin area M. FIG. Also in this case, when the corresponding point corresponding to the feature point included in the matching image SMA1 cannot be extracted from the matching image SMA2 corresponding to the matching image SMA1, the divided images are combined in the direction of generation of parallax, and the matching image is enlarged. Therefore, finally, the corresponding point corresponding to the feature point included in the matching image SMA1 can be extracted from the matching image SMA2 corresponding to the matching image SMA1.

In addition, in this embodiment, although image data was demonstrated as being a set of image 61 and the image 62 image | photographed by the digital camera 70, it is not limited to this, The image data was formed by digitizing a satellite photograph. It may be a digital image obtained by scanning an image or a photograph taken by a general analog camera.

In addition, in this embodiment, although the correspondence point in the image 62 corresponding to the characteristic point in the image 61 was extracted using the area correlation method, it is not limited to this, Another method, for example, Japanese Patent Application Laid-Open You may use the method of 8-16930.

9 is a block diagram showing an example of a physical configuration when the stereo image processing apparatus 10 is mounted on a computer. The stereo image processing apparatus 10 according to the present embodiment can be realized by a hardware configuration similar to that of a general computer apparatus. As shown in FIG. 9, the stereo image processing apparatus 10 includes a control unit 21, a main memory unit 22, an external storage unit 23, an operation unit 24, a display unit 25, and an input / output unit 26. It is provided. The main memory unit 22, the external storage unit 23, the operation unit 24, the display unit 25, and the input / output unit 26 are all connected to the control unit 21 via the internal bus 20.

The control unit 21 is composed of a CPU (Central Processing Unit) or the like, and performs stereo matching processing in accordance with the control program 30 stored in the external storage unit 23.

The main memory section 22 is composed of a RAM (Random-Access Memory) or the like. The main memory section 22 loads the control program 30 stored in the external storage section 23 and is used as a work area of the control section 21.

The external storage unit 23 is composed of a nonvolatile memory such as a flash memory, a hard disk, a digital versatile disc random-access memory (DVD-RAM), a digital versatile disc re-writable (DVD-RW), and controls the above process. The control program 30 to be made to the 21 is stored in advance, and according to the instruction of the control unit 21, the data stored in the control program 30 is supplied to the control unit 21, and the control unit 21 is provided. Store the data supplied from

The operation unit 24 is composed of a pointing device such as a keyboard and a mouse and the like, and an interface device for connecting the keyboard and pointing device to the internal bus 20. Through the operation unit 24, an instruction such as input of image data, transmission and reception, an instruction of an image to be displayed, and the like are input and supplied to the controller 21.

The display unit 25 is composed of a cathode ray tube (CRT), a liquid crystal display (LCD), or the like, and displays an image or a result of stereo matching.

The input / output unit 26 is composed of a wireless transceiver, a wireless modem or a network termination device, and a serial interface or a local area network (LAN) interface connected thereto. Via the input / output unit 26, image data can be received and the result of the calculation can be transmitted.

In the stereo image processing apparatus 10 shown in FIG. 1, the data input unit 11, the image extraction unit 12, the image division unit 13, the matching image setting unit 14, the correspondence point extraction unit 15, and matching The control of the control program 30 includes the control unit 21, the main storage unit 22, the external storage unit 23, and the processing of the miss detection unit 16, the divided image combiner 17, and the three-dimensional information calculation unit 18. It executes by processing using the operation part 24, the display part 25, the input / output part 26, etc. as a resource.

In addition, the above-mentioned hardware configuration and flowchart are an example, and can be changed and modified arbitrarily.

Center for performing processing of the stereo image processing apparatus 10 including the control unit 21, the main memory unit 22, the external storage unit 23, the operation unit 24, the input / output unit 26, the internal bus 20, and the like. The part to be realized can be realized using a normal computer system, regardless of a dedicated system. For example, by storing and distributing a computer program for performing the above operation on a computer-readable recording medium (flexible disc, CD-ROM, DVD-ROM, etc.), and installing the computer program on the computer, You may comprise the stereo image processing apparatus 10 which performs the process of. In addition, the stereo image processing apparatus 10 may be configured by storing the computer program in a storage device of a server device on a communication network such as the Internet and downloading it by a normal computer system.

In addition, when the function of the stereo image processing apparatus 10 is realized by sharing the OS (operating system) and the application program or by cooperating with the OS and the application program, only the application program portion is stored in the recording medium or the storage device. You may save it.

It is also possible to superimpose a computer program on a carrier wave and distribute it through a communication network. For example, the computer program may be posted on a bulletin board system (BBS) on a communication network, and the computer program may be distributed through a network. Then, the computer program may be started, and under the control of the OS, the computer program may be executed in the same manner as in other application programs to execute the above process.

In addition, about this application, the priority based on Japanese Patent Application 2008-300220 is claimed, and the specification, patent claim, and drawing of Japanese Patent Application 2008-300220 are referred to in this specification by reference.

10: stereo image processing device
11: data input unit
12: image extraction unit
13: image segmentation unit
14: matching image setting unit
15: correspondence point extraction unit
16: matching miss detection unit
17: split image combiner
18: three-dimensional information calculation unit
21:
22: main memory
23: external storage
24: control panel
25:
26: input and output unit
30: control program
61, 62: burns
61a, 62a: Extraction image
61a (m, n), 62a (m, n): block image
SMA1, SMA2: Matching Images
70: digital camera
71, 72: dry matter

Claims (7)

An image processing apparatus which performs stereo matching processing on a first image and a second image photographed from different positions,
Each of the first image and the second image is divided into a plurality of images, and the divided image of the first image and the margin area around it, the divided image of the second image corresponding to the divided image, and the margin around it A setting unit that sets the area to the first matching image and the second matching image, respectively;
An extraction unit for extracting a corresponding point included in the second matching image corresponding to a point included in the first matching image;
A combination obtained by combining another divided image adjacent to the divided image to each of the divided image set to the first matching image and the divided image set to the second matching image, when the extracting unit cannot extract the corresponding point. A resetting unit for resetting the image and the margin area around the image to the first matching image and the second matching image, respectively;
A calculation unit for calculating three-dimensional information of each point included in the matching image of the first image and a corresponding point included in the matching image of the second image
An image processing apparatus having a. The method of claim 1,
And the reset unit combines adjacent divided images in a direction in which parallax occurs between the first image and the second image. As an image processing method for performing stereo matching processing on a first image and a second image photographed from different positions,
Each of the first image and the second image is divided into a plurality of images, and the divided image of the first image and the margin area around it, the divided image of the second image corresponding to the divided image, and the margin around it A setting step of setting the areas as the first matching image and the second matching image, respectively;
An extraction step of extracting a corresponding point included in the second matching image corresponding to a point included in the first matching image;
In the case where the corresponding point cannot be extracted in the extraction step, the respective divided images set as the first matching image and the divided images set as the second matching image are obtained by combining another divided image adjacent to the divided image. A resetting process of resetting the combined image and the margin area around the combined image to the first matching image and the second matching image, respectively,
Calculating step of calculating three-dimensional information of each point included in the matching image of the first image and the corresponding point included in the matching image of the second image
Image processing method comprising a. The computer divides each of the first image and the second image photographed from different positions into a plurality of images, and divides the first image and a margin area around the first image and a second image corresponding to the divided image. A setting unit which sets the divided image and the margin area around it as the first matching image and the second matching image, respectively;
An extraction unit for extracting a corresponding point included in the second matching image corresponding to a point included in the first matching image;
A combination obtained by combining another divided image adjacent to the divided image to each of the divided image set to the first matching image and the divided image set to the second matching image, when the extracting unit cannot extract the corresponding point. A resetting unit for resetting the image and the margin area around the image to the first matching image and the second matching image, respectively;
An operation unit for calculating three-dimensional information of each point included in the matching image of the first image and a corresponding point included in the matching image of the second image
A computer-readable recording medium having recorded thereon a program to function as a computer. delete delete delete

Images