JP2003058869A - Correspondent point searching method and photographing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate erroneous correspondent as little as possible in correspondent point search and to perform exact corresponding. SOLUTION: In the method for finding mutual correspondent points in an object area concerning two images having a parallax while respectively having an object area and a background area, concerning the background area adjacent to the object area, the luminance of a background device arranged behind an object is changed so that the luminance can be close to the luminance of the object area, a multi-resolution image is prepared concerning two images to find the correspondent points on the basis of an image photographed after the change, points corresponding to the images of respective resolution are found in order from the image of the lowest resolution, and the result is used for finding the point corresponding to the image of the next resolution.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a corresponding point searching method for two images of an object captured in stereo, and an imaging system used for the method. In particular, it is effective when performing a corresponding point search using a multi-resolution image. The present invention is used to generate three-dimensional shape data of an object.

[0002]

2. Description of the Related Art Conventionally, a plurality of cameras having a parallax are acquired by using a plurality of cameras, and a calibration technique, a corresponding point search technique,
Also, there is known a three-dimensional generation device that obtains three-dimensional shape data by using a three-dimensional reconstruction technique. The gradient method (optical flow) and the correlation method have heretofore been known as corresponding point search techniques used for this purpose.

In order to accurately search for corresponding points, a method using a multi-resolution image has been proposed. In this method, the original image is set as the image with the highest resolution, and images with lower resolution are sequentially created in a hierarchical manner. At each resolution, the image is composed of image blocks, which are unit areas for search. The search for corresponding points is performed in order from the image with the lowest resolution. At that time, the search result in the image of a certain resolution is 1
Propagated to the initial value of the search for the image of the hierarchical resolution.

[0004]

In the corresponding point search method using a multi-resolution image, when a low-resolution search is performed and the difference in luminance between the object in the image and the background is large, the error is erroneous. Correspondence may occur.

That is, when searching using a multi-resolution image, the brightness of each image block is compared at each resolution to make correspondence. As the brightness of each image block, for example, the average brightness (brightness) of the pixels included in each image block is used. The image block size increases as the resolution decreases. In the image block on the boundary between the target region, which is the region where the corresponding point is desired, and the background region, which is the other region, the brightness of the image block greatly depends on the area and the brightness of the background region.

Therefore, even if the image blocks originally correspond to each other, the brightness may greatly differ due to the difference in the background area occupation area due to the parallax. In such a case, the image blocks may not be correctly corresponded, which may result in an incorrect correspondence.

In the method using the multi-resolution image, when there is an erroneous correspondence at a low resolution, it is propagated to a high resolution, so that an erroneous correspondence occurs in a wide range, and three-dimensional shape data is accurately generated. I couldn't do it.

The present invention has been made in view of the above problems, and eliminates erroneous correspondences in the corresponding point search as much as possible.
The purpose is to make an accurate correspondence.

[0009]

A method according to the present invention is a method for obtaining mutually corresponding points in a target area for two images having a parallax each having a target area and a background area. For the background area adjacent to the area, the brightness of the background device arranged behind the subject is changed so that the brightness is close to the brightness of the target area, and a corresponding point is obtained based on the image captured after the change. .

If necessary, a multi-resolution image is created for the two images, corresponding points are obtained for each resolution image in order from the one with the lowest resolution, and the result is obtained for the next resolution image. It is used when finding corresponding points.

The image pickup system according to the present invention includes an image pickup apparatus for taking an image of an object from a plurality of different viewpoints together with the background thereof, and the brightness of the background is changed independently of the brightness of the object when the image is taken. And a display device that displays information about at least the brightness of the boundary between the object and the background imaged by the imaging device.

[0012] Preferably, the display device displays an image captured by the image capturing device, and when the boundary portion is designated in the image, at least two adjacent border portions are designated at the designated boundary portion. Displays information about the brightness of block areas.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing the whole of a three-dimensional generation apparatus 1 to which the corresponding point searching method according to the present invention is applied, and FIG. 2 is a block diagram showing a configuration example of a processing apparatus 5.

In FIG. 1, the three-dimensional generator 1 includes a multi-lens camera 3 for photographing an object (subject) Q, a background device 4 installed as a background behind the object Q, and a processor 5. Become. Although not shown, the object Q
An illumination device for illuminating the object is provided in front of the object Q.

The multi-lens camera 3 is a twin-lens camera in which two digital cameras 3k and 3l are connected to each other and integrated. The cameras 3k and 3l each include an optical system and an image sensor. In the following, the reference numerals of the members and elements relating to the one camera 3k may be attached with "k", and the reference numerals of the members and elements relating to the other camera 3l may be attached with "l".

By pressing the release button of the multi-lens camera 3 and photographing the object Q once, two images FD with a parallax of the object Q are obtained. In addition, the image FD captured by the image sensor is continuously output in real time until the release button is pressed, that is, before shooting.

The multi-lens camera 3 has a memory for storing a large number of images FD, a memory for storing camera parameters, a liquid crystal display for displaying the images FD, various operation buttons such as a release button, a flash, an input / output interface, and control. Unit, a power supply unit, and the like are provided.

An image FD before photographing, an image FD photographed,
Alternatively, the edited image of the image FD can be transferred to the processing device 5 and other external devices via the input / output interface. Further, it is also possible to input the image FD to an external device by removing the memory storing the image FD.

The processing device 5 performs various processes described later on the image FD obtained by the multi-lens camera 3, performs corresponding point searching process, performs calibration, and generates a three-dimensional model ML. The processing device 5 also issues various commands to the background device 4 to control the background device 4. In addition,
The function of the processing device 5 may be built in the multi-lens camera 3 and the multi-lens camera 3 may generate the three-dimensional model ML.

As shown in FIG. 2, the processing device 5 includes a device body 10, a magnetic disk device 11, and a medium drive device 1.
2, a display device 13, a keyboard 14, a mouse 15 and the like.

The apparatus main body 10 includes a CPU, a RAM, and an RO.
M, video RAM, input / output port, and various controllers. Various functions described below are realized by the CPU executing programs stored in the RAM, the ROM, and the like.

The magnetic disk device 11 has an OS (Operat)
ing System), a modeling program PR for generating a three-dimensional model ML, other programs, an input image (two-dimensional image data) FD, a generated three-dimensional model ML, other data, etc. are stored. . These programs and data are loaded into the RAM of the apparatus body 10 at appropriate times.

The modeling program PR includes programs for multiplexing processing, associating processing, three-dimensional reconstruction processing, mapping processing, and other processing.

The medium drive device 12 is a CD-ROM.
(CD), floppy disk FD, magneto-optical disk,
The semiconductor memory HM such as Compact Flash (registered trademark) or other recording media is accessed to read or write data or programs. An appropriate drive device is used according to the type of recording medium. The modeling program PR described above can also be installed from these recording media. The image FD and the like can also be input via the recording medium.

On the display surface HG of the display device 13,
Various data, image FD, multi-resolution image,
Image block, brightness value, background instruction message, image of process by modeling program PR, generated 3
The dimensional model ML and other data or images are displayed.

The keyboard 14 and the mouse 15 are used for designating an automatic mode or a manual mode for controlling the background device 4 for the image FD displayed on the display device 13, designating a display position of luminance information, and the like. It is used to input various data to the apparatus main body 10 or give a command.

The processing device 5 can be constructed by using a personal computer or a workstation. The programs and data described above can also be acquired by receiving them via the network NW.

FIG. 3 is a block diagram functionally showing the processing device 5. In FIG. 3, the processing device 5 includes an image input unit 5
1, a multiplexing unit 52, a brightness calculation unit 53, a background instruction unit 54,
It has an associating unit 55, a three-dimensional reconstruction unit 56, a display unit 57, a block designating unit 58, a background designating unit 59, and the like.

The image input unit 51 inputs the image FD obtained by the multi-lens camera 3 and stores it in the memory. Multiplexer 52
Creates a multi-resolution image including a plurality of layers based on the input image (original image) FD. Brightness calculation unit 5
3 calculates the brightness and the brightness difference of the image block or the like designated by the block designating section 58 for the image of each layer. The background instruction unit 54 outputs a command for controlling the brightness of the background device 4 to the control device 34 based on the calculated brightness and the brightness difference.

The associating unit 55 searches for corresponding points based on the multi-resolution image. When the corresponding point search is performed by using the brightness, the brightness is adjusted in advance so that the brightness of the target area of the image matches the brightness of the background area. Color matching point search
If done with, the colors are pre-adjusted to match. Details will be described later.

The three-dimensional reconstruction unit 56 performs three-dimensional reconstruction using the searched corresponding points to generate three-dimensional shape data DT. The display unit 57 displays the image of each layer, the brightness value of the image block, the message from the background instruction unit 54, and the like on the display device 13.

The block designating section 58 designates an image block in which luminance is to be displayed in the image, based on an operation input from the operation input section such as the keyboard 14 and the mouse 15. The background designating unit 59 outputs a command for adjusting the brightness of the background device 4 to the control device 34 based on the operation input.

It is to be noted that not only the brightness but also the color and the pattern, if necessary, an instruction, an instruction or a message is output and displayed on the display surface HG. 4 shows the background device 4 according to the first embodiment, FIG. 5 shows the background device 4B according to the second embodiment, FIG. 6 shows the background device 4C according to the third embodiment, and FIG. FIG. 8 is a diagram showing an example of a pattern PN.

In FIG. 4, the background device 4 is attached to the housing 31 having a rectangular parallelepiped box shape and having an opening, a plurality of fluorescent lamps 32 provided inside the housing 31, and the opening of the housing 31. It consists of a diffusion plate 33.

The housing 31 is made of an opaque material, and a reflecting plate for reflecting the light of the fluorescent lamp 32 is provided inside. The diffusion plate 33 diffuses the light of the fluorescent lamp 32 over one surface to make the light uniform. Diffusion plate 33
A variety of achromatic colors having different densities (that is, different gray levels), different colors, different patterns, etc. are prepared and can be used by replacing them.

An identification code is provided on each of the diffusion plates 33. The identification code of the diffusion plate 33 to be used may be displayed on the display surface HG of the processing device 5. In that case, the diffusion plate 33 indicated on the display surface HG may be replaced.

The background device 4 is controlled by the control device 34. That is, the control device 34 controls the brightness (brightness) of the fluorescent lamp 32 in accordance with the command output from the processing device 5. For example, the brightness is continuously adjusted from the darkest state (low brightness state) to the brightest state (high brightness state).

In FIG. 5, the background device 4B comprises a transmissive screen 41 and a projection device (projector) 42. The projection device 42 projects light onto the screen 41. At that time, the brightness of the projected light can be variably adjusted. By providing a filter on the exit surface of the projection device 42, it is possible to project light of various colors and patterns.

In FIG. 6, the background device 4C is an LED array display device 45 in which a large number of light emitting diodes are arranged in a matrix. By adjusting the brightness of the light emitting diode, the brightness (brightness) of the background can be adjusted. Further, as the light emitting diode, for example, R,
When the three colors of G and B are arranged so as to form one pixel, various colors can be emitted by adjusting the ratio of their luminance. It is also possible to display various patterns.

As shown in FIGS. 7 (A) to 7 (C), various patterns PN1 to PN3 having a check pattern of a high density portion and a low density portion are used. These patterns PN1 to PN3 are used as the diffusion plate 33, used as a filter of the projection device 42, or displayed on a display surface such as an LED array.

Since the three-dimensional position (distance) of the background area EH is fixed by using the pattern PN, the target area ET and the background area EH can be easily separated. In addition, as the background device 4, it is possible to use a display device including various display devices such as a CRT, an LCD, and a PDP. Further, a sheet-like material made of cloth, paper, wood, synthetic resin or the like may be used, and the illumination light may be applied from the outside.

FIG. 8 shows the display device 13 of the processing device 5.
It is a figure which shows the example of the image FD displayed on. FIG. 8 (A)
In (C), the head of the person is displayed as the object Q. Two images FD output from the multi-lens camera 3
Among them, one image FD is shown. By operating the operation input unit, the two images FD can be switched and displayed.

In each image FD, the face is the target area ET and the portion other than the head is the background area EH. Figure 8
(A) to (C) are obtained by changing the brightness KD of the background device 4 with respect to the same target, and FIG. 8 (A) is the darkest.
(C) is the brightest.

The brightness KD of the background device 4 can be manually adjusted by the user or automatically controlled. In the case of manual operation, the user operates the operation input unit according to the instruction displayed on the display surface HG of the display device 13. A command value is output from the processing device 5 to the background device 4 according to a user operation. Thereby, the brightness KD of the background device 4 can be arbitrarily adjusted between the minimum and the maximum. In the case of automatic control, the brightness of the background area EH is controlled to be close to the brightness of the target area ET. Each of the manual and automatic modes can be selected by the operation input unit.

FIG. 9 is a diagram for explaining a multi-resolution image. FIG. 9A shows an image (original image) FDa before or after photographing. The image FDa has, for example, 1
It is composed of 024 × 1024 pixels. In the case of the original image, the corresponding point search is performed for each pixel. That is, each pixel is a search unit area. A unit area for searching in each layer is called an “image block GB”. In the original image, each pixel becomes an image block GB.

The image FDb in FIG. 9B is an image having a resolution several layers lower than that of the image FDa. In this image FDb, the number of image blocks GB is 12 × 12. Therefore, one image block GB consists of about 85 × 85 pixels.

The image FDc in FIG. 9C has a resolution one layer lower than that of the image FDb. In this image FDc, the number of image blocks GB is 6 × 6. Therefore, one image block GB consists of about 170 × 170 pixels.

The image FDd in FIG. 9 (d) is one layer lower than the image FDc and has the lowest resolution. In this image FDd, the number of image blocks GB is 3 × 3.
Therefore, one image block GB is 341 × 34.
It consists of about one pixel.

By operating the keyboard 14 or the mouse 15, the image FD of any hierarchy can be selected and displayed on the display surface HG. It is possible to display an image FD of an arbitrary layer before shooting and adjust the brightness KD of the background device 4 in that state.

FIG. 10 is a diagram showing how the background device 4 is adjusted for the brightness KD, and FIG. 11 is a diagram showing an example of the brightness KD displayed in the image block GB. In FIG. 10, two image blocks GB in the vicinity of the position indicated by the cursor CS are displayed while the image FDc is displayed. In each image block GB, the brightness (brightness) KD of the image block GB is displayed numerically. When the operation input unit is operated to bring the cursor CS to the edge portion of the target area ET, that is, the boundary portion KB between the target area ET and the background area EH, the brightness KD of the boundary portion KB and the adjacent background area EH is real time. Is displayed.

In the example of FIG. 11A, the brightness KDa of the left image block GBa, that is, the boundary portion KB is "6".
1 ", the brightness KDb of the right image block GBb, that is, the background area EH is" 86 ". In the present embodiment, these numerical values are displayed in the range of 0-255. In the case of this example, the brightness difference KDS (= | KDb−KDa |) between the two image blocks GBa and GBb is “25”, which is considerably large. If the brightness difference KDS is large, the brightness gradients of the corresponding parts of the two images FD with parallax may differ greatly, and the corresponding point search may not be performed accurately. The reason will be described later.

Therefore, the user manually adjusts the brightness KD of the background device 4 so that the brightness difference KDS becomes small.
At that time, the cursor CS is moved along the edge portion of the target area ET, and the image block GB is moved along the edge portion.
a, GBb are displayed. The user grasps what the brightness difference KDS is for the entire edge portion, that is, the entire boundary portion KB between the target area ET and the background area EH. Then, the brightness of the background device 4 is adjusted by operating the operation input unit to give an instruction so that the brightness difference KDS becomes smaller as a whole.

Even if the brightness difference is small at one place of the boundary portion KB, it does not necessarily become small at other places. Therefore, for example, "5" or less, or "1" is set so that the brightness difference KDS is equal to or less than a constant value at all points of the boundary portion KB.
Adjust so that it becomes 0 ”or less. In the example of FIG. 11B, the image blocks GBa and GBb are “58”, respectively.
"62", and the difference is "4", which is sufficiently small.

As described above, the user operates the operation input section to adjust the brightness of the background device 4 while watching the brightness value displayed on the display surface HG. Further, on the display surface HG, a message about the identification code of the diffusion plate 33 to be used for the background device 4, the color, the type of pattern, etc. may be displayed. For example, as a message instructing color,
"The optimal background for this subject is R = ○○, G = XX, B
= △△ 」is displayed. The user adjusts the brightness or color of the background device 4 according to the message, selects an appropriate pattern PN, or, if necessary, the diffuser plate 33.
Or replace the filter.

In the automatic mode, the brightness or color of the background device 4 is automatically controlled so that the brightness difference KDS is minimized. Next, the overall flow of processing and control by the three-dimensional shape generation device 1 will be described with reference to the flowchart.

FIG. 12 is a flow chart showing the overall flow of processing by the three-dimensional shape generating apparatus 1, FIG. 13 is a flow chart showing manual background setting, and FIG. 14 is a flow chart showing automatic background setting.

In FIG. 12, after setting the background (# 11), the image FD is obtained by photographing with the multi-lens camera 3 (# 12). Corresponding point search is performed on the image FD (# 13), and three-dimensional reconstruction is performed using the search result (#
14).

In FIG. 13, the image FD is first input to set the background (# 21). For the target area, the brightness and color (hue) of each image block are calculated (# 22). For the background area, the brightness and color of each image block are calculated (# 23). Based on them, a message for instructing adjustment of the background device 4 is displayed on the display surface HG (# 24). In response to the message, the user adjusts the background device 4, changes the background such as replacing the diffusion plate 33 (# 25).

By repeating the process or operation of FIG. 13 a plurality of times, the background area EH and the target area E are
The brightness difference KDS from T is further reduced. In FIG. 14, steps # 31 to 33 are the same as step # 31 described above.
21 to 23. Calculate the control value (# 34),
The background is controlled (# 35). These processes are repeated as needed.

In the above-described embodiment, the brightness of the background area EH is adjusted or controlled based on the brightness of the entire target area ET. Next, based on the brightness of a specific area of the target area ET. An embodiment for performing adjustment or control will be described.

FIG. 15 shows a monitor window ME set in the target area ET, and FIG. 16 shows a monitor window M.
9 is a flowchart showing setting of a background using E. Note that FIG. 16 is a subroutine of step # 11 of FIG.

In FIG. 15, the image FDe before photographing is displayed on the display surface HG. In the image FDe, a rectangular monitor window ME having an appropriate size is set at a substantially central portion of the target area ET. The setting of the monitor window ME may be performed by the user or may be automatically performed.

In FIG. 16, after inputting the image FD (# 41), the monitor window ME is set (# 4).
2). A brightness value representing the monitor window ME is calculated (# 43). In calculating the brightness value, for example,
The average brightness of the pixels in the monitor window ME is calculated. The background device 4 is controlled using the calculated brightness value (# 4
4). At that time, the brightness of the background device 4 is controlled so as to be close to the calculated brightness value. Further, the processing of FIG. 16 is repeated.

As a result, the brightness KD of the background area EH becomes close to the brightness KD of the target area ET, and erroneous correspondence can be prevented in the corresponding point search. Therefore, it is possible to generate accurate three-dimensional shape data without any chipping due to incorrect correspondence.

Here, the reason why the corresponding point search is not accurately performed when the luminance difference KDS is large will be described. Figure 1
7 is a diagram showing two images FDL and FDR having parallax, FIG. 18 is a diagram showing luminance gradients of the images FDL and FDR, and FIG.
Reference numeral 9 denotes each image FDL, FD after the brightness of the background area EH is changed.
It is a figure which shows the brightness gradient of R.

Note that FIG. 18 (A) and FIG. 19 (A)
Indicates the luminance gradient for the fourth image block GB in the column (Y column). 18B and 19B
Is a row (X) in FIG. 18 (A) or FIG. 19 (A).
The column) shows an enlarged luminance gradient from the fourth image block GBc to the fifth image block GBd.

In each image FD, the vertical column (Y column) is the fourth column.
Attention is paid to the fourth and fifth image blocks GBc and GBd whose number and row (X column) are the same. The image block GBc includes both the target area ET and the background area EH, and exists at the boundary between them. The image block GBd is the background area E.
Exists in H.

In the image block GBc, the right image F
The image FDL on the left side of the DR has a larger area of the background region EH. In the state shown in FIG. 17, the background area EH
Since the brightness is low, the brightness is considerably lower (darker) than the brightness of the target area ET.

Therefore, under the influence of the brightness difference between the target area ET and the background area EH, the brightness difference KDS between the two images is large in the image block GBc as shown in FIG. 18 (A).

In FIG. 18B, the brightness value in the image block GBc is set to the same position in the two images, and the brightness gradients thereof are shown for easy comparison. According to this, between the two images, the image block GBc to the image block GBd
The difference in the brightness gradient to is obvious.

In this state, the image blocks GBc, GB
Although d should be associated with each other between the two images, there is a possibility that the association will not be performed due to a large difference in the luminance gradient between them. In that case, it will be an incorrect response.

Therefore, when the background area EH is increased in brightness to be bright, the brightness gradient from the image block GBc to the image block GBd becomes almost the same as shown in FIGS. 19 (A) and 19 (B), which is correct. Correlation is performed.

In the above-described embodiment, the configuration, shape, number, image contents, processing contents and order of the processing device 5, the background device 4, or the three-dimensional shape generating device 1 as a whole or each part are as follows. It can be changed appropriately according to the spirit of the invention.

[0074]

According to the present invention, erroneous correspondence can be eliminated as much as possible in the corresponding point search, and accurate correspondence can be performed.

[Brief description of drawings]

FIG. 1 is a diagram showing an entire three-dimensional generation device to which a corresponding point search method according to the present invention is applied.

FIG. 2 is a block diagram illustrating a configuration example of a processing device.

FIG. 3 is a block diagram functionally showing the processing device.

FIG. 4 is a diagram showing a background device according to the first embodiment.

FIG. 5 is a diagram illustrating a background device according to a second embodiment.

FIG. 6 illustrates a background device according to a third embodiment.

FIG. 7 is a diagram showing an example of a pattern.

FIG. 8 is a diagram showing an example of an image displayed on a display device of the processing device.

FIG. 9 is a diagram illustrating a multi-resolution image.

FIG. 10 is a diagram showing how the background device is adjusted in brightness.

FIG. 11 is a diagram showing an example of luminance displayed in an image block.

FIG. 12 is a flowchart showing the overall flow of processing by the three-dimensional shape generation device.

FIG. 13 is a flowchart showing manual background setting.

FIG. 14 is a flowchart showing automatic background setting.

FIG. 15 is a diagram showing a monitor window set in a target area.

FIG. 16 is a flowchart showing setting of a background using a monitor window.

FIG. 17 is a diagram illustrating two images with parallax.

FIG. 18 is a diagram showing a brightness gradient of each image.

FIG. 19 is a diagram showing a brightness gradient of each image after the brightness of the background area is changed.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 3D shape generation device 3 Multi-lens camera (imaging device) 4, 4B, 4C Background device (background setting device) 5 Processing device 13 Display device (display device) 34 Control device (luminance control unit) 53 Luminance calculation unit (luminance Acquisition section) 54 Background instruction section GB, GBa, GBb, GBc, GBd Image block (block area) ET Target area EH Background area FD image PN pattern

Continued front page    (72) Inventor Satoru Hirose             2-3-3 Azuchi-cho, Chuo-ku, Osaka-shi, Osaka Prefecture               Osaka International Building Minolta Co., Ltd. (72) Inventor Shinichiro Hirama             Hiroo ON Bi 5-19-9 Hiroo, Shibuya-ku, Tokyo             Le Gen Co., Ltd. (72) Inventor Kazushi Suzuki             Hiroo ON Bi 5-19-9 Hiroo, Shibuya-ku, Tokyo             Le Gen Co., Ltd. F term (reference) 2F065 AA53 BB05 CC16 DD00 EE00                       FF05 FF42 GG03 GG07 GG18                       HH02 JJ03 JJ05 JJ26 LL49                       NN01 QQ03 QQ24 QQ36 SS02                       SS03 SS13                 5B057 AA20 BA02 CA01 CA08 CA12                       CA16 CB01 CB08 CB12 CB16                       CC03 CE08 CE11 CH01 CH11                       DA20 DB02 DB06 DB09 DC32

Claims (6)

[Claims] 1. A method for obtaining mutually corresponding points in a target area for two images having parallax each having a target area and a background area, the brightness of the background area being adjacent to the target area. Is changed so as to be close to the luminance of the target area, and the luminance of the background device arranged behind the subject is changed, and the corresponding point is obtained based on the image captured after the change. 2. When a multi-resolution image is created for the two images, corresponding points are obtained for the images of respective resolutions in order from the image with the lowest resolution, and the result is obtained for corresponding points of the image of the next resolution. The corresponding point searching method according to claim 1, which is used for. 3. A photographing device for photographing an object together with its background from a plurality of different viewpoints, and a background setting capable of varying the brightness of the background independently of the brightness of the object at the time of photographing. An imaging system, comprising: a device; and a display device that displays information regarding at least the brightness of a boundary portion between the object and the background imaged by the imaging device. 4. The display device displays an image photographed by the photographing device, and when the boundary portion is designated in the image,
The imaging system according to claim 3, wherein information about the brightness of at least two block areas adjacent to each other in the designated boundary portion is displayed. 5. A photographing device for photographing an object together with its background from a plurality of different viewpoints, and a background setting capable of varying the luminance of the background independently of the luminance of the object at the time of photographing. A device, a brightness acquisition unit that acquires information about brightness of at least a boundary portion between the object and the background imaged by the imaging device, and the brightness of the background approaches the brightness of the object. A brightness control unit for controlling the background setting device; 6. The photographing system according to claim 3, wherein the background setting device is capable of setting one or a plurality of patterns as the background.