JP2001141417A - Parallax amount correction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parallax amount correction device for measuring a distance up to an object based on an object image. SOLUTION: This parallax amount correction device is provided with an image pickup unit 20 provided with a lens unit 22 and a light receiving part 30, an image pickup control unit 40 for controlling a zoom or the like, a processing unit 60 for processing digital image data, a display unit 100 for displaying digital images and an operation unit 110 operated by a user. Further, in the lens unit, a parallax amount generated in two object images at the time of being image-formed on a light receiving plane is provided with image height characteristics to be nonlinear to the distance to the object and a parallax amount correction unit for outputting image data obtained by performing sampling by non-uniform density corresponding to the image height characteristics so as to cancel the nonlinearity of the parallax amount is provided.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a camera and a laboratory system. The invention particularly relates to cameras and lab systems for measuring distance.

[0002]

2. Description of the Related Art In a general camera, an f.tan.theta. Lens designed to photograph a subject image without distortion is used. Further, there are f.theta. Lenses (fisheye lenses) as lenses designed to have a wider angle of view than the human eye. On the other hand, conventionally, there is a technique called stereo photography in which the same subject is photographed from a plurality of viewpoints. The distance from the camera to the subject can be determined based on the apparent difference (parallax) between the two images obtained by this technique. This is similar to the principle of human binocular stereovision.

[0003]

However, f · tan θ
When performing stereo imaging using a lens, an equal parallax plane in which the amount of parallax generated on an image plane when images are formed through a plurality of viewpoints is a plane orthogonal to the optical axis of the lens. However, this parallax plane is not an exact equidistant plane from the camera. Further, when stereo imaging is performed using an f · θ lens, the isoparallax plane is a spherical surface passing through a plurality of viewpoints and a subject, but the isoparallax plane is also not an accurate equidistant plane from the camera. As described above, it is difficult to measure the distance accurately regardless of which lens is used for stereo photography.

[0004] Therefore, an object of the present invention is to provide a parallax amount correcting apparatus which can solve the above-mentioned problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous embodiments of the present invention.

[0005]

According to a first aspect of the present invention, there is provided an apparatus for acquiring image data of a subject image obtained from a lens system having first and second viewpoints. The lens system has an image height characteristic in which the amount of parallax generated between the two subject images when formed on a light receiving plane is non-linear with respect to the distance from the shooting position to the subject. A parallax amount correction unit that outputs the image data obtained by sampling at an uneven density according to the image height characteristic of the lens system so as to cancel the non-linearity of the parallax amount.

[0006] The parallax amount correction unit is configured to make the parallax amount generated between the two subject images in the image data non-uniform according to an image height characteristic of the lens system so as to be proportional to a distance from a shooting position to a subject. The image data obtained by sampling at an appropriate density may be output. The parallax amount correction unit is configured to have a non-uniform density such that parallax amounts generated in two subject images when a subject on an equidistant plane from a shooting position are formed through first and second viewpoints are equal. And outputting the image data obtained by sampling at.

When the azimuth between a normal line passing through the photographing position and a line connecting the photographing position and the subject is represented by θ, the parallax amount correcting unit determines that the subject on the plane equidistant from the photographing position is the first object. The parallax amount occurring in the two subject images when formed on the light receiving plane through the second viewpoint,

(Equation 3) The image data obtained by sampling at an uneven density such that the parallax is corrected by the correction function h that satisfies the following may be output.

The parallax correction unit may output the image data obtained by sampling at an optically non-uniform density according to the image height characteristic of the lens system. The parallax amount correction unit may include a plurality of light receiving elements arranged at an optically unequal density according to an image height characteristic of the lens system. Among the plurality of light receiving elements, the light receiving elements farther from the optical axis of the lens system may be arranged at a lower density. The parallax amount correction unit may include a plurality of light receiving elements arranged on a light receiving surface curved at a degree corresponding to an image height characteristic of the lens system. The parallax amount correction unit,
The light from the lens system may be exposed to an image pickup body curved at a degree corresponding to the image height characteristic of the lens system.

The parallax correction unit may output the image data obtained by sampling at an electrically non-uniform density according to the image height characteristic of the lens system. The parallax correction unit has a plurality of light receiving elements arranged at an optically uniform density, and converts an analog signal of the image data obtained from these light receiving elements into an electric signal according to an image height characteristic of the lens system. The image data may be converted into a digital signal of the image data obtained by sampling at a non-uniform density. The image processing apparatus may further include a depth distribution information generation unit that generates depth distribution information based on the parallax amount.

In a second embodiment of the present invention, an image input section for inputting first image data of a subject image obtained from a lens system having first and second viewpoints is formed on a light receiving plane. The first image data of the subject image obtained from the lens system having an image height characteristic in which the amount of parallax generated between the two subject images when formed is nonlinear with respect to the distance from the shooting position to the subject. And an image processing unit that converts the image data into second image data so as to cancel the nonlinearity of the parallax amount.

[0011] The image processing unit may convert the first image data into the second image data in which a parallax amount generated between the two subject images is proportional to a distance from a shooting position to the subject. . The image processing unit may be configured to convert the first image data into a first object at an equidistant plane from a shooting position.
The image data may be converted to the second image data such that the parallax amounts generated in the two subject images when formed through the second viewpoint are equal. The image processing unit sets an azimuth formed by a normal passing through the shooting position and a line connecting the subject and the shooting position equidistant from the shooting position to θ, and sets an image height characteristic of the lens system to g ( θ), the first
The parallax amount generated between the two subject images in the image data of

(Equation 4) The first image data may be converted to the second image data by correcting the parallax with a correction function h that satisfies the following.
The image processing apparatus may further include a depth distribution information generation unit that generates depth distribution information based on the parallax amount.

The above summary of the present invention does not list all of the necessary features of the present invention, and a sub-combination of these features may also be an invention.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described through embodiments of the present invention. However, the following embodiments do not limit the invention according to the claims and are described in the embodiments. Not all combinations of features are essential to the solution of the invention.

The parallax amount correcting apparatus of the present invention described below is used for general image photographing in each embodiment, and can measure the distance to a subject using the photographed image. Hereinafter, embodiments of the present invention will be described. The parallax correction device according to the first embodiment is a digital camera as an example. FIG. 1 shows a configuration of a digital camera 10 according to the embodiment. The digital camera 10 mainly includes an imaging unit 20, an imaging control unit 40, a processing unit 60, a display unit 100, and an operation unit 110.

The imaging unit 20 includes a mechanism member and an electric member related to photographing and image formation. Imaging unit 20
First, a lens unit 2 that captures and processes images
2, aperture 24, shutter 26, optical LPF (low-pass filter) 28, light receiving unit 30, and image signal processing unit 32
including. The lens unit 22 may be configured by a focus lens, a zoom lens, or the like. With this configuration, a subject image is formed on the light receiving surface of the light receiving unit 30. Electric charges are accumulated in each sensor element (not shown) of the light receiving unit 30 according to the light quantity of the formed subject image (hereinafter, the electric charges are referred to as “accumulated charges”). The accumulated charge is read out to a shift register (not shown) by a read gate pulse, and is sequentially read out as a voltage signal by a register transfer pulse. The light receiving section 30 has a plurality of light receiving elements (not shown) arranged on a light receiving surface. As the light receiving element, for example, a CCD (charge coupled element) is used.

Since the digital camera 10 generally has an electronic shutter function, a mechanical shutter is not essential. In order to realize the electronic shutter function, the light receiving unit 30 is provided with a shutter drain via a shutter gate. When the shutter gate is driven, accumulated charges are swept out to the shutter drain. By controlling the shutter gate, it is possible to control the time for accumulating charges in each light receiving element, that is, the shutter speed.

An electric signal output from the light receiving section 30, that is, an analog signal is color-separated into R, G, and B components by an image pickup signal processing section 32, and the white balance is adjusted first.
Subsequently, the imaging signal processing unit 32 performs gamma correction, sequentially A / D converts the R, G, and B signals at necessary timing, and obtains digital image data (hereinafter, simply referred to as “digital image data”) obtained as a result. ) Is output to the processing unit 60.

The imaging unit 20 further includes a finder 3
4 and a strobe 36. The finder 34 may be equipped with an LCD (not shown). In this case, various information from the main CPU 62 and the like described later can be displayed in the finder 34. The strobe 36 functions by emitting light when energy stored in a capacitor (not shown) is supplied to the discharge tube 36a.

The imaging control unit 40 includes a position control unit 24
0, an imaging system CPU 50, a distance measurement sensor 52, and a photometry sensor 54. The position control section 240 has driving means such as a stepping motor. Release switch 1 described later
When the button 14 is pressed, the distance measurement sensor 52 measures the distance to the subject, and the photometry sensor 54 measures the brightness of the subject. The measured distance data (hereinafter simply referred to as “distance measurement data”) and the subject luminance data (hereinafter simply referred to as “photometry data”) are sent to the imaging system CPU 50. Imaging system C
The PU 50 adjusts the zoom magnification through control by the position control unit 240 based on shooting information such as the zoom magnification specified by the user. The imaging system CPU 50 determines a shutter speed based on an integrated value of RGB digital signals of one image frame, that is, AE information. The imaging system CPU 50 controls the electronic shutter function of the light receiving unit 30 according to the determined value.

The image pickup system CPU 50 also controls light emission of the strobe 36 based on photometric data. When the user instructs the capture of an image, the light receiving unit 30 starts charge accumulation,
After a lapse of the shutter time calculated from the photometric data, the accumulated charge is output to the imaging signal processing unit 32.

The processing unit 60 includes the imaging signal processing unit 32
Process digital image data received from The processing here includes the process of processing digital images,
It also includes simply storing data in memory. The processing unit 60 includes a main CPU 62 that controls the entire digital camera 10, particularly the processing unit 60 itself, and a memory control unit 64, a YC processing unit 70, an optional device control unit 74, a compression / decompression processing unit 78, I /
An F portion 80 is provided. The main CPU 62 exchanges necessary information with the imaging system CPU 50 by serial communication or the like. The operation clock of the main CPU 62 is provided from a clock generator 88. Clock generator 88
Provides clocks of different frequencies to the imaging system CPU 50 and the display unit 100, respectively.

The main CPU 62 is provided with a character generator 84 and a timer 86. The timer 86 is backed up by a battery and always counts the date and time.
From this count value, information about the shooting date and time and other time information are given to the main CPU 62. The character generation unit 84 generates character information such as a shooting date and time and a title, and the character information is appropriately combined with the captured image.

The memory control unit 64 includes a nonvolatile memory 66
And the main memory 68. Non-volatile memory 66
The digital camera 1 includes an EEPROM (electrically erasable and programmable ROM), a flash memory, and the like, and stores setting information by a user and adjustment values at the time of shipment.
The data to be held even while the power supply of 0 is off is stored. The non-volatile memory 66 may have a main CPU
62, a boot program and a system program may be stored. On the other hand, the main memory 68 generally has a DR
It is composed of a relatively inexpensive and large-capacity memory like an AM. The main memory 68 functions as a frame memory for storing data output from the imaging unit 20,
It has a function as a system memory for loading various programs and a function as a work area. The nonvolatile memory 66 and the main memory 68 exchange data with each unit inside and outside the processing unit 60 via the main bus 82.

The YC processing section 70 performs YC conversion on the digital image data, and outputs a luminance signal Y and a color difference (chroma) signal B-
Generate Y, RY. The luminance signal and the color difference signal are temporarily stored in the main memory 68 by the memory control unit 64.
The compression / decompression processing unit 78 sequentially reads out the luminance signal and the color difference signal from the main memory 68 and compresses them. The data thus compressed (hereinafter simply referred to as “compressed data”) is written to a memory card, which is a type of the optional device 76, via the optional device control section 74.

The processing unit 60 further includes an encoder 72
With. The encoder 72 receives a luminance signal and a color difference signal, converts them into a video signal (NTSC or PAL signal), and outputs the video signal from a video output terminal 90. When a video signal is generated from data recorded in the optional device 76, the data is first supplied to the compression / decompression processing unit 78 via the optional device control unit 74. Subsequently, the data subjected to the necessary expansion processing by the compression / expansion processing unit 78 is converted into a video signal by the encoder 72.

The optional device control unit 74 generates necessary signals between the main bus 82 and the optional device 76, performs logical conversion, or converts voltage according to the signal specifications recognized by the optional device 76 and the bus specifications of the main bus 82. And so on. The digital camera 10 may support a magnetic recording medium such as a floppy disk in addition to the above-described memory card as the optional device 76.
Further, the digital camera 10 may support, for example, a standard I / O card conforming to PCMCIA as the optional device 76. In that case, the optional device control unit 7
4 may be configured by a PCMCIA bus control LSI or the like.

The communication I / F unit 80 is a digital camera 10
Communication specifications supported by, for example, USB, RS-232
It performs control such as protocol conversion according to specifications such as C and Ethernet. The communication I / F unit 80 includes a driver IC as necessary, and communicates with an external device including a network via a connector 92. In addition to such standard specifications, for example, a unique I / F
May be adopted.

The display unit 100 includes an LCD monitor 10
2 and an LCD panel 104. These are the LCD driver monitor driver 106, panel driver 10
8 respectively. LCD monitor 102
Is provided on the back of the camera with a size of, for example, about 2 inches, and displays a current shooting and playback mode, a zoom ratio for shooting and playback, a remaining battery level, a date and time, a screen for mode setting, a subject image, and the like. . The LCD panel 104 is, for example, a small black-and-white LCD provided on the upper surface of the camera and has an image quality (FINE).
/ NORMAL / BASIC etc.), information such as strobe light emission / non-light emission, standard number of recordable images, number of pixels, battery capacity, etc. are simply displayed.

The operation unit 110 includes mechanisms and electrical members necessary for a user to set or instruct the operation of the digital camera 10 and its mode. The power switch 112 determines whether the power of the digital camera 10 is turned on or off. The release switch 114 has a two-stage pressing structure of half pressing and full pressing. As an example, when the shutter button is pressed halfway, the distance measuring operation and the photometric operation are locked, and when the shutter button is pressed fully, the captured image is captured. You.
The zoom switch 118 determines a zoom magnification. The operation unit 110 may receive a setting using a rotary mode dial, a cross key, or the like in addition to these switches, and these are collectively referred to as a function setting unit 116 in FIG. Examples of operations or functions that can be specified by the operation unit 110 include “file format”, “special effects”, “printing”, “determine / save”, and “display switching”.

FIG. 2 is a functional block diagram of the digital camera 10. The digital camera 10 includes a lens unit 22, a parallax correction unit 300, and a depth distribution information generation unit 302. The lens unit 22 has a lens system 200 and a lens driving unit 304. Lens system 2
00 forms the subject image through the first and second viewpoints. The lens driving unit 304 moves the lens system 200 between the first viewpoint and the second viewpoint. Accordingly, the lens system 200 can form a subject image viewed from each position of the first viewpoint and the second viewpoint.

The parallax amount correction unit 300 includes a light receiving unit 30
And an imaging signal processing unit 32. The light receiving unit 30 receives external light through the lens system 200. The subject images viewed from two viewpoints can be obtained by the photoelectric conversion operation of the light receiving unit 30. Parallax occurs between these subject images due to apparent differences. The subject image is captured by the imaging signal processing unit 3
2, the image data is converted into digital image data.

The parallax correction unit 300 corrects the parallax generated in the two subject images formed on the light receiving plane by a predetermined method, so that the parallax in the image data is reduced to the distance from the shooting position to the subject. Make proportional. The parallax amount correction unit 300 samples the subject image at an uneven density according to the image height characteristic of the lens system such that the parallax amount generated between the two subject images in the image data is proportional to the distance from the shooting position to the subject. I do. When a subject located at an equidistant plane from the shooting position is imaged, the parallax amounts generated in the two subject images in the image data output by the parallax correction unit 300 are equal. Depth distribution information generator 30
2 generates depth distribution information based on the amount of parallax. In another embodiment, the lens drive unit 304 for moving the lens system 200 may not be provided, and the lens system 200 may be provided for each of the first and second viewpoints.

As described above, in the present embodiment, the parallax amounts generated for the subjects located at the same distance from the shooting position are equal, so that the distance to the subject can be easily calculated by obtaining the parallax amount.

Next, a stereo using an f-tan θ lens
The shooting will be described. FIG. 3 uses an f-tan θ lens.
It is a figure which shows the stereo imaging which was performed. Lens system 2 shown in this figure
02 and 204 are subjects within the field of view of these lens systems.
Is the image height, f is the focal length, and camera position is
And the angle between the line connecting the subject and the optical axis (hereinafter referred to as “azimuth
Corner. " ) Is θ, y = f · tan θ and
Image height characteristics. Lens with such characteristics
Is a common camera because it can shoot images without distortion
Often used for. Then, the lens systems 202 and 20
When shooting in stereo using 4
Equal vision where the amount of parallax generated on the imaging plane when imaging is equal
The difference plane is a plane orthogonal to the optical axis of the lens systems 202 and 204.
Becomes In other words, each point A on this isoparallax plane₀,
A₃₀, And A₆₀The amount of parallax corresponding to
₀, Id₃₀, And Id₆₀Then Id₀= Id
₃₀= Id ₆₀Becomes

The term "lens system" used in the claims and the detailed description of the invention means one or more optical elements having a single optical axis. For example,
Each of the lens system 202 and the lens system 204 in FIG. 3 is one “lens system”,
Each of the lenses 204 may be formed by joining a plurality of lenses.

Here, each point A ₀ , A ₃₀ , and A ₆₀
Are different from each other to a base point B (an intermediate point between the position of the lens system 202 and the position of the lens 204). That is, as the position of point A _theta away from the optical axis, the distance from point A _theta to the point B is far away. As described above, the parallax plane is not a plane equidistant from the camera position (base point B).

Distance measurement by stereo photographing using an f-tan θ lens does not pose a problem if it is used for, for example, an autofocus function of a camera. This is because, in autofocus, focus adjustment is performed using the distance to the equi-parallax plane (focal plane). However, since the amount of parallax obtained through the f-tan θ lens is not proportional to the distance to the subject, calculating an accurate distance using the f-tan θ lens is complicated.

Next, stereo photography using the f · θ lens will be described. FIG. 4 is a diagram showing stereo photography using an f · θ lens. Lens systems 206 and 208 shown in FIG.
Has an image height characteristic of y = f · θ, where y is the image height of an image formed by imaging a subject in the field of view of these lens systems, f is the focal length, and θ is the azimuth angle. When photographing is performed using a lens having such characteristics, a so-called omnidirectional image having an angle of view close to 180 ° is obtained. Then, when stereo imaging is performed using the lens systems 206 and 208, the isoparallax plane is a spherical surface centered on a predetermined point O that is equidistant from the position of the lens system 206 and the position of the lens system 208. That is, the parallax amounts corresponding to the respective points A ₀ , A ₃₀ , and A ₆₀ on the isoparallax plane are represented by Id ₀ , Id, respectively.
₃₀ and Id ₆₀ , Id ₀ = Id ₃₀ = I
d ₆₀ .

The isoparallax plane in the present embodiment is a locus of a point where the binocular parallax becomes zero when compared with human eyes, and is generally called a horopter. This equiparallax plane is an equidistant plane from the point O, but is not an equidistant plane from the camera position. As shown in the figure, the distance from each of the points A ₀ , A ₃₀ , and A ₆₀ to the base point B (an intermediate point between the position of the lens system 206 and the position of the lens system 208) is different. Then, the farther the position of the point A _θ is from the optical axis, the more the point A
The distance from _θ to point B is short. As described above, the parallax plane is not a plane equidistant from the camera position (base point B).

As in the case of using the f.tan.theta. Lens, distance measurement by stereo photographing using the f.theta. Lens does not pose a problem as long as it is used for the autofocus function of the camera. However, since the amount of parallax obtained through the f · θ lens is not proportional to the distance to the subject, the calculation of an accurate distance using the f · θ lens is complicated.

Next, stereo shooting using a lens having an image height characteristic in which the amount of parallax is proportional to the distance to the subject will be described.
FIG. 5 is a diagram illustrating stereo shooting using a lens having an image height characteristic in which the amount of parallax is proportional to the distance to the subject. When this lens is used, an equi-parallax plane in which the amount of parallax generated on the image plane when the outside world is imaged through the first and second viewpoints is substantially equal to an equidistant plane from the parallax amount correction device. Matches. A lens system (lens systems 210 and 212) is provided for each of the first and second viewpoints.

The equidistant surface from the parallax amount correcting apparatus of the present embodiment may be, for example, a spherical surface having a base point at an intermediate point between the first viewpoint and the second viewpoint. In FIG. 5, the lens system 2
An arc having a radius r around a base point B, which is an intermediate point between the center point (point C) of the lens system 212 and the center point (point D) of the lens system 212, indicates an equidistant plane. The lens systems 210 and 212 have characteristics such that the equi-parallax plane substantially matches the equidistant plane. Here, the angle between the optical axis and the line connecting the shooting position (point B) and the subject (hereinafter, referred to as “azimuth angle”) is θ. Also, a point _Aθ indicating the position of the subject on the equi-parallax plane and the lens system 210
Angle (hereinafter, referred to as "parallax angle".) And a line connecting the lines and points A _theta and a lens system 212 connecting to take the alpha _theta. Assuming that the parallax angles corresponding to the respective points A ₀ , A ₃₀ , and A ₆₀ on the equi-parallax plane are α ₀ , α ₃₀ , and α ₆₀ , respectively, these parallax angles α _θ are in a relationship of α ₀ > α ₃₀ > α ₆₀ . Becomes That is, the farther the point A is from the optical axis, the smaller the parallax angle α _θ is. Hereinafter, the relationship between the parallax angle α _θ and the azimuth angle θ will be clarified.

When a coordinate system (z, y) having a point B as an origin (0, 0) as shown in FIG.
The coordinates of points C and D indicating the position of 2 are C = (0, -d),
D = (0, d). The coordinates of the point A ₀ are given by A ₀ = (−
r, 0). Point A _θ (where θ = 0 ° to 90 °)
Is a point on an arc of radius r, the coordinates of the point A _θ are A _θ
= (− R · cos θ, r · sin θ). Therefore, the vector from the subject to each viewpoint is

(Equation 5) Becomes Then, the parallax angle α _θ is expressed by the following equation based on a formula for obtaining an angle formed by two vectors.

(Equation 6)

Here, when the numerator and denominator on the right side are expanded,

(Equation 7) Becomes Therefore, α (θ) can be expressed by a detailed equation such as the following equation.

(Equation 8)

FIG. 6 is a graph showing the relationship between α (θ) and θ. As shown in the graph, the value of α (θ) is r
The band is determined by the ratio of d and d, but in each case the curve is similar. Then, when normalized by α (θ) when θ = 0, that is, by α (0), α _N (θ) = α (θ)
/ Α (0). This α _N (θ) is a parallax angle α with respect to the azimuth angle θ of the subject when α (0) is set to 1.
Shows the rate of decrease.

FIG. 7 shows α _N (θ) using d as a parameter.
6 is a graph showing a relationship between and θ. As shown in the drawing, the smaller the parameter d is, the closer the curve is to the curve of cos θ. Assuming that r >> d, α _N (θ)
≒ cos θ. That is, as the azimuth angle θ increases, the parallax angle α decreases like cos θ. If this lens system is provided with an image height characteristic for correcting a decrease in the parallax angle α, an image height characteristic in a form of integrating the correction amount 1 / α _N (θ) may be provided. That is, the correction amount α _G (θ)
Is

(Equation 9) Becomes That is, the image height characteristic of this lens system is represented by y = f
-It can be expressed as α _G (θ). Further, if the distance to the subject and the distance between the two viewpoints are in a relationship of r >> d, cosx can be substituted for α _N (x).

FIG. 8 is a graph showing image height characteristics of various lenses. In this graph, the vertical axis represents the image height y and the horizontal axis represents the azimuth angle θ, and the parameters are the image height characteristics of various lenses.
The various lenses used for the parameters are an f · tan θ lens, an f · θ lens, and an f · α _G (θ) lens. In the range of 0 ° <θ <90 °, the value of the image height y of this lens system is f · θ <y <f · tan θ. The curve of y = f · α _G (θ) indicating the image height characteristic of this lens system is located between the curve of y = f · tan (θ) and the curve of y = f · θ. That is, this lens system is f · ta
It has image height characteristics between the nθ lens and the f · θ lens.

FIG. 9 is a graph showing the relationship between the azimuth and the amount of parallax in stereo photography using an α _G (θ) lens. First, the azimuth is set to θ, and the image height of the subject image when a subject located at an equidistant plane from the shooting position is formed is y _1.
And The first quadrant shows the relationship between the azimuth angle θ and image height y _1. The curve shows the image height characteristics of the α _G (θ) lens. Since the focal length f is a constant, it is not particularly shown in this graph. Image heights y _1A , azimuths θ _A , θ _B , θ _C
y _1B and y _1C correspond to each other.

The second quadrant shows the image height y _1, the amount of parallax that occurs two object images on the light receiving plane (hereinafter, referred to as. "No correction parallax amount") the relationship between. Here, assuming that the increment of the azimuth based on a certain azimuth is Δθ, the difference between the origin, θ _A , θ _B , and θ _{C on} the horizontal axis (θ axis) of the first quadrant is Δθ (Δθ _A , Δθ _B , Δθ _C ). Similarly,
When the increment of the image height _{y 1} and [Delta] y _1, the first quadrant of the vertical axis (y
₍₁ axis), the difference between the origin, y _1A , y _1B , y _1C is Δ
y ₁ (y _1A , y _1B , y _1C ). And
Origin in the second quadrant of the horizontal axis _{(y 2 axes), y 2A, y
2B, Δy 2 (y} 2A is a difference between _{y 2C, y 2B, y
2C} ) shows the uncorrected parallax amount for each azimuth. y
_{2A, y} 2B, _{y 2C are, Δy 2A, y 2B, y} 2C can be said that the integral value for taking the value as described above.

In the third quadrant, the uncorrected parallax amount and the image data
(Hereinafter referred to as “corrected vision)
The difference is called. ). The vertical axis of the third quadrant (y
₃Origin at y), y_3A, Y_3B, Y_3CThe difference between
Δy₃(Y_3A, Y_3B, Y_3C) For each azimuth
The corrected parallax amount is shown. y_3A, Y_3B, Y_3CIs
Δy_3A, Y_3B, Y_3CTakes the above value
It can be said that the integral value of. Between the uncorrected parallax amount and the corrected parallax amount
When the relationship is represented by a correction function h, y₃= H (y ₂) And
You. Corrected vision by developing from the first quadrant to the third quadrant
The relationship between the difference amount and the azimuth angle can be understood. Corrected parallax amount Δy₃Is
It is constant regardless of the value of the azimuth angle θ. That is, the shooting position
After correction when imaging an object on the equidistant plane from
The differences are equal.

Here, since the α _G (θ) lens has the property that the amount of parallax is proportional to the distance to the subject due to the image height characteristics of the lens, it is not necessary to correct the amount of parallax. Therefore, no correction parallax amount [Delta] y ₂ will be equal. The correction function h _{becomes h (y 2)} = y 2 or _{h (y 2) = k ·} y 2. In the second quadrant, y ₂ must proportional to azimuth angle theta. Therefore, the relationship between y ₁ and y ₂ can be represented by the inverse function of the relationship between θ and y _1. Since relationship between the position angle theta to image height _{y 1} is _{y 1 = α G (θ)} , a relational expression between _{y 1} and _{y 2} are, _{y 2} = alpha _G becomes ^-1 _{(y 1).} The curve in the second quadrant shows y ₂ = α _G ⁻¹ (y ₁ ).

[0052] The _{y 2} is proportional to theta is the image height characteristics of the lens used is only when _{y 1 = α G (θ)} . That is, when a lens whose image height characteristic is y ₁ = α _G (θ) is not used, y ₂ is not proportional to θ. Therefore,
The relational expression between y ₁ and y ₂ in the second quadrant is always y ₂ = α _G ⁻¹ (y
₁ ).

[0053] relationship between the y ₃ θ are as follows.

(Equation 10)

Here, h is h (y ₂ ) = y ₂ or k
If the function is y ₂ (k is a constant), y ₃ is proportional to θ. That is, the amount of parallax when the α _G (θ) lens is used is constant without correction.

FIG. 10 shows a stereo using an f · θ lens.
6 is a graph showing a relationship between an azimuth angle and a parallax amount in shooting.
You. The relationship shown in the first to third quadrants is
It is the same as FIG. The straight line in the first quadrant is f · θ
The image height characteristics of the lens are shown. Since the focal length f is fixed,
Vertical axis y in the first quadrant₁The value of y is for convenience₁Use / f
I have. The relationship between the uncorrected parallax amount and the corrected parallax amount is
Expressed by the number h, y ₃= H (y₂).

As described above, the relationship between y ₁ and y ₂ in the second quadrant is y ₂ = α _G ⁻¹ (y ₁ ) even when the f · θ lens is used. Since the image height characteristic of the f · theta lens is linear, the image height y ₁ is proportional to the azimuth angle theta.
The relational expression of y ₂ = α _G ⁻¹ (y ₁ ) converts y ₁ that is linear with respect to θ into y ₂ that is a non-linear value with respect to θ.
The correction function h in the third quadrant is a function that cancels the nonlinearity in the second quadrant and obtains a linear value for the azimuth angle θ again. Therefore, the correction function h is given by y ₂ = α
K · α _G (y ₂ ), which is the inverse function of _G ^{− 1} (y ₁ ), (k
Is a constant). In the graph, the curve in the third quadrant is line-symmetric to the curve in the second quadrant.

[0057] relationship between the y ₃ θ are as follows.

[Equation 11]

Here, h is h {α _G ⁻¹ (f · θ)}.
= K ₁ · θ, where k ₁ is a constant, y ₃
Is proportional to θ. h (y ₂ ) = k ₂ · α
_G (y ₂ ), where k ₂ is a constant, h ｛α
^{_{G -1 (f · θ)}}} = k 2 · α G {α G - 1 (f ·
_{θ)} = k 3 · f} · θ = k 3 · θ, the _{(k 3} is a constant). Therefore, the parallax amount correction unit 300 is h
The amount of parallax may be corrected based on a function h that satisfies (y ₂ ) = k ₂ · α _G (y ₂ ).

FIG. 11 is a graph showing the relationship between the azimuth and the amount of parallax in stereo photography using an f-tan θ lens. The relationship shown in the first to third quadrants is the same as in FIG. The curve in the first quadrant is f
The image height characteristics of the tan θ lens are shown. Since the focal length f is fixed, y ₁ / f is used for the value of the vertical axis y ₁ in the first quadrant for convenience. When the relationship between uncorrected parallax amount _{y 2} and after correction parallax amount _{y 3} shows a correction function _h, y 3 = h (y
₂ )

As in the case of using the f · θ lens, the relationship between the image height y ₁ and the amount of parallax y ₂ in the second quadrant is f · ta
Even when an nθ lens is used, y ₂ = α _G ⁻¹ (y
₁ ). Correction function h in the third quadrant, a curve _y 3 = h _{(y 2)} shown in the graph. relationship between the y ₃ θ are as follows.

(Equation 12)

Here, h is h ｛α _G ⁻¹ (f · tan
θ)｝ = k · θ, where k is a constant, y
₃ is proportional to θ. Therefore, the parallax amount correcting unit 300 calculates h {α _G ⁻¹ (f · tan θ)} = k
The parallax amount may be corrected based on the function h that becomes θ.

FIG. 12 is a graph showing the relationship between the azimuth and the amount of parallax in stereo photography using a lens having an arbitrary image height characteristic. The relationship shown in the first to third quadrants is the same as in FIG. Since the focal length f is a constant, it is not particularly shown in this graph. The image height characteristic of this lens is y ₁ = g (θ). The curve in the first quadrant shows the image height characteristic y ₁ = g (θ) of this lens.
When the relationship between the uncorrected parallax amount and the corrected parallax amount indicated by the correction function h, the _{y 3 = h (y 2)} .

The relationship between the image height y ₁ and the amount of parallax y ₂ in the second quadrant is y ₂ = α _G ⁻¹ (y ₁ ). The correction function h in the third quadrant is a curve y ₃ = h shown in the graph.
(Y ₂ ). relationship between the y ₃ θ are as follows.

[0064]

(Equation 13) Here, h is h [α _G ^-1 {g (θ)}] = k · θ,
(K is a constant) If the function to be, so that the y ₃ is proportional to theta. Therefore, the parallax amount correction unit 300 may correct the parallax amount based on the function h that satisfies h [α _G ⁻¹ {g (θ)}] = k · θ.

FIG. 13 shows a configuration of a lens system and a light receiving section in this embodiment. The parallax correction unit 300 in the present embodiment has a plurality of light receiving elements arranged at an optically uneven density according to the image height characteristics of the lens system.
Among the plurality of light receiving elements, the light receiving elements farther from the optical axis of the lens system are arranged with lower density.

The lens system 200 has an image height characteristic in which the amount of parallax generated between two subject images when formed on the light receiving plane is non-linear with respect to the distance from the shooting position to the subject. By arranging a plurality of light receiving elements at unequal density in the light receiving unit 30, a correction is made to cancel the non-linearity of the parallax caused by the image height characteristic of the lens system 200. The light receiving unit 30 can sample image data at an uneven density according to the image height characteristics of the lens system 200. The correction of the parallax amount by the light receiving unit 30 is represented by a correction function h. That is, the function h is the lens system 200
H [α _G ^-1 g, where g (θ) is the image height characteristic of
(Θ)｝] = k · θ, where k is a constant.

The amount of parallax between two subject images in the image data obtained by the light receiving unit 30 is proportional to the distance from the shooting position to the subject. When a subject on an equidistant plane from the shooting position is imaged through the first and second viewpoints, the parallax amounts generated in the two subject images are equal. in this way,
Since the parallax amount is proportional to the distance to the subject, the distance to the subject can be easily obtained based on the parallax amount.

In another embodiment, the function of the depth distribution information generating unit for obtaining the distance from the amount of parallax may be provided not in the camera but in the lab system. in this case,
The digital camera 10 performs correction of the amount of parallax, and the lab system generates depth distribution information based on the obtained image.

Next, a second embodiment will be described. FIG.
Shows the configuration of the lens system, the light receiving unit, and the imaging signal processing unit in the present embodiment. In the present embodiment, the arrangement of the plurality of light receiving elements in the light receiving section is made uniform, and the subject image is sampled at an uneven density in the A / D conversion by the imaging signal processing section 32. The image height characteristics of the lens system 200 are the same as in the first embodiment. The imaging signal processing unit 32 performs a correction for canceling the non-linearity of the parallax amount in the image height characteristic of the lens system 200.

The image pickup signal processing section 32 extracts an analog signal output from the light receiving section 30 at an uneven density according to the image height characteristic of the lens system 200. The correction of the amount of parallax by the imaging signal processing unit 32 is represented by a correction function h. That is, assuming that the image height characteristic of the lens system 200 is g (θ), the function h is
h [α _G ^-1 {g (θ)}] = k · θ, where k is a constant. In another embodiment, the imaging signal processing unit 32 extracts an analog signal output at a uniform pixel density from the light receiving unit, and outputs a digital signal at an uneven pixel density according to the image height characteristic of the lens system. Is also good.

The amount of parallax between two subject images in the image data output from the image pickup signal processing unit 32 is proportional to the distance from the shooting position to the subject. As described above, since the parallax amount is proportional to the distance to the subject, the distance to the subject can be easily obtained based on the parallax amount.

In another embodiment, the function of the depth distribution information generating unit for obtaining the distance from the amount of parallax may be provided not in the camera but in the lab system. in this case,
The digital camera 10 performs correction of the amount of parallax, and the lab system generates depth distribution information based on the obtained image.

Next, a third embodiment will be described. FIG.
Shows the configuration of the lens system and the light receiving unit in the present embodiment. The parallax amount correction unit according to the present embodiment includes a plurality of light receiving elements arranged on a light receiving surface curved at a degree corresponding to the image height characteristic of the lens system 200. As a result, image data sampled at a substantially uneven density is obtained, and the non-linearity of the parallax amount in the image height characteristic of the lens system 200 is canceled.

The correction of the amount of parallax by the light receiving unit 30 is represented by a correction function h. That is, the function h is the lens system 200
H [α _G ^-1 g, where g (θ) is the image height characteristic of
(Θ)｝] = k · θ, where k is a constant. The amount of parallax between two subject images in the image data output from the imaging signal processing unit 32 is proportional to the distance from the shooting position to the subject.

In another embodiment, the function of the depth distribution information generating unit for obtaining the distance from the amount of parallax may be provided not in the camera but in the lab system. in this case,
The digital camera 10 performs correction of the amount of parallax, and the lab system generates depth distribution information based on the obtained image. As still another embodiment, similarly to the light receiving unit 30 in the present embodiment, the parallax amount correction unit 300 sensitizes the imaging body curved to a degree corresponding to the image height characteristic of the lens system 200 with light from the lens system. May be. As the imaging body, for example, a silver halide film is used. In this case, the lab system generates depth distribution information based on image data obtained by scanning a silver halide film or the like. The amount of parallax generated between two subjects in an imaging body such as a silver halide film is proportional to the distance from the shooting position to the subjects.

Next, a fourth embodiment will be described. The parallax amount correction device in the present embodiment is a lab system as an example. FIG. 16 is a functional block diagram of the laboratory system according to the present embodiment. This lab system includes an image input unit 400, an image processing unit 410, and a depth distribution information generation unit 420.

The image input unit 400 inputs image data of a subject image obtained from a lens system having first and second viewpoints. When the image input unit 400 inputs image data from a recording medium, as the image input unit 400, various reading devices according to the standard of the recording medium are used. For example, when the image input unit 400 inputs image data from a floppy disk, MO, or CD-ROM, a floppy drive, MO drive, or CD drive is used as the image input unit 400. The image input unit 400 may be, for example, an interface that captures image data from a digital camera. In this case, the image input unit 400 has communication specifications supported by the digital camera, for example, USB, RS-232.
The protocol conversion according to C or the like is performed. Further, for example, the image input unit 400 may be an interface that receives image data from a network. In this case, protocol conversion is performed according to the communication specifications of the network, for example, TCP / IP.

The image processing unit 410 determines whether the amount of parallax generated in the two subject images when formed on the light receiving plane is from a lens system having an image height characteristic that is nonlinear with respect to the distance from the shooting position to the subject. The obtained first image data is received from the image input unit 400. The image processing unit 410 converts the first image data into second image data in which the nonlinearity of the amount of parallax has been canceled. The image processing unit 410 includes a parallax amount extraction unit 412 and a parallax amount correction unit 414. The parallax amount extraction unit 412 extracts a parallax amount between two subject images received from the image input unit 400. Parallax amount correction unit 414
Converts the first image data into second image data in which the amount of parallax is proportional to the distance from the shooting position to the subject, thereby correcting the non-linearity of the amount of parallax received from the parallax amount extraction unit 412. .

The correction of the amount of parallax by the parallax amount correction unit 414 is represented by a correction function h. That is, assuming that the image height characteristic of the lens system 200 is g (θ), the function h is h [α _G
⁻¹ {g (θ)}] = k · θ, where k is a constant. The amount of parallax generated between two subject images in the image data output from the parallax correction unit 414 is proportional to the distance from the shooting position to the subject.

The depth distribution information generation section 420 generates depth distribution information based on the corrected parallax amount received from the parallax amount correction section 414.

In another embodiment, the parallax correction device may be applied to a digital camera as an example.
In this case, the image input unit 400 corresponds to the imaging unit 20 in FIG. 1, and the image processing unit 410 and the depth distribution information generation unit 420 correspond to the processing unit 60.

As described above, according to the first to fourth embodiments, the parallax amount generated in two subject images when a subject on the equidistant plane from the photographing position is formed is equal. , The distance to the subject can be easily calculated.

Further, the parallax correction apparatus of the present embodiment can measure the distance to the subject with high accuracy regardless of the image height characteristics of the lens used.

The parallax correction device of the present embodiment can be applied to various devices and functions that require accurate measurement of the distance to an object. For example, the present invention may be applied to a camera having a function of adjusting the intensity of illumination light based on information on a distance to an object. For example, the present invention may be applied to a robot having an arm mechanism whose position is controlled based on information on a distance to an object. For example, the present invention may be applied to a coating apparatus that controls the strength of injecting a paint based on information on the distance to an object.

As described above, the present invention has been described using the embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be made to the above embodiment. It is apparent from the description of the appended claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

As is apparent from the above description, according to the present invention, the distance to the subject can be measured based on the subject image.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a digital camera 10. FIG.

FIG. 2 is a functional block diagram of the digital camera 10.

FIG. 3 is a diagram showing stereo shooting using an f · tan θ lens.

FIG. 4 is a diagram illustrating stereo shooting using an f · θ lens.

FIG. 5 is a diagram illustrating stereo shooting using the parallax correction device according to the first embodiment.

FIG. 6 is a graph showing a relationship between α (θ) and θ.

FIG. 7 is a graph showing the relationship between α _N (θ) and θ using d as a parameter.

FIG. 8 is a graph showing image height characteristics of various lenses.

FIG. 9 is a graph showing the relationship between the azimuth and the amount of parallax in stereo shooting using an f · α _G (θ) lens.

FIG. 10 is a graph showing the relationship between the azimuth and the amount of parallax in stereo shooting using an f · θ lens.

FIG. 11 is a graph showing the relationship between the azimuth angle and the amount of parallax in stereo shooting using an f-tan θ lens.

FIG. 12 is a graph showing the relationship between the azimuth and the amount of parallax in stereo shooting using a lens having an arbitrary image height characteristic.

FIG. 13 is a diagram illustrating a configuration of a lens system and a light receiving unit according to the first embodiment.

FIG. 14 is a diagram illustrating a configuration of a lens system, a light receiving unit, and an imaging signal processing unit according to a second embodiment.

FIG. 15 is a diagram illustrating a configuration of a lens system and a light receiving unit according to a third embodiment.

FIG. 16 is a functional block diagram of a laboratory system according to a fourth embodiment.

[Explanation of symbols]

 Reference Signs List 10 digital camera 20 imaging unit 22 lens unit 30 light receiving unit 32 imaging signal processing unit 40 imaging control unit 60 processing unit 100 display unit 110 operation unit 200 lens system 300 parallax amount correction unit 302 depth distribution information generation unit 304 lens driving unit 400 image Input unit 410 Arithmetic processing unit 412 Parallax amount extraction unit 414 Parallax amount correction unit 420 Depth distribution information generation unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) // H04N 13/02 G06F 15/62 415 5C061 F term (reference) 2F065 AA02 AA06 DD03 EE00 FF05 JJ00 JJ02 JJ25 LL00 LL21 QQ00 QQ03 QQ13 QQ17 QQ24 QQ38 2F112 AC03 BA06 CA02 CA12 DA28 FA03 FA21 FA38 5B057 CD12 DA07 DB03 DC02 5C022 AA13 AB23 AB24 AB66 AB68 AC00 5C054 EA01 FC03 FC15 FD02 FD07 HA05 5C061 AA21 AB02

Claims (17)

[Claims] 1. An apparatus for acquiring image data of a subject image obtained from a lens system having first and second viewpoints, wherein the lens system is configured to form two image data when formed on a light receiving plane. The amount of parallax generated in the subject image has an image height characteristic that is non-linear with respect to the distance from the shooting position to the subject, and an image height characteristic corresponding to the image height characteristic of the lens system is set so as to cancel the non-linearity of the parallax amount. A parallax correction apparatus, comprising: a parallax correction unit that outputs the image data obtained by sampling at a uniform density. 2. The parallax amount correction unit according to an image height characteristic of the lens system so that a parallax amount generated between two subject images in the image data is proportional to a distance from a shooting position to a subject. The parallax amount correction apparatus according to claim 1, wherein the image data obtained by sampling at an uneven density is output. 3. The parallax amount correction unit is configured to equalize parallax amounts generated in two subject images when a subject on an equidistant plane from a shooting position is imaged through first and second viewpoints. And outputting the image data obtained by sampling at a non-uniform density.
Or the parallax amount correction device according to 2. 4. The parallax amount correction unit, when an azimuth between a normal line passing through a shooting position and a line connecting the shooting position and the subject is θ, an object on an equidistant plane from the shooting position is determined. The parallax amount generated in the two subject images when formed on the light receiving plane through the first and second viewpoints is expressed by the following equation. The parallax amount correcting apparatus according to any one of claims 1 to 3, wherein the image data obtained by sampling at an uneven density such that the parallax is corrected by a correction function h that satisfies is output. 5. The apparatus according to claim 1, wherein the parallax amount correction unit outputs the image data obtained by sampling at an optically uneven density according to an image height characteristic of the lens system. 5. The parallax amount correction device according to any one of 4. 6. The parallax amount correction unit according to claim 1, wherein the parallax amount correction unit includes a plurality of light receiving elements arranged at an optically uneven density according to an image height characteristic of the lens system. A parallax amount correction device according to any of the above. 7. The parallax correction device according to claim 6, wherein, among the plurality of light receiving elements, the light receiving elements farther from the optical axis of the lens system are arranged with a lower density. 8. The apparatus according to claim 1, wherein the parallax amount correction unit includes a plurality of light receiving elements disposed on a light receiving surface curved at a degree corresponding to an image height characteristic of the lens system.
6. The parallax correction device according to any one of claims 1 to 5. 9. The apparatus according to claim 1, wherein the parallax amount correction unit sensitizes the image pickup body curved at a degree corresponding to an image height characteristic of the lens system with light from the lens system. The parallax amount correction device according to any one of the above. 10. The parallax correction unit outputs the image data obtained by sampling at an electrically non-uniform density according to an image height characteristic of the lens system. 5. The parallax amount correction device according to any one of 4. 11. The parallax amount correction unit includes a plurality of light receiving elements arranged at an optically uniform density, and converts an analog signal of the image data obtained from these light receiving elements into an image height of the lens system. The parallax correction device according to claim 10, wherein the image data is converted into a digital signal of the image data obtained by sampling at an electrically non-uniform density according to a characteristic. 12. The parallax amount correction apparatus according to claim 1, further comprising a depth distribution information generation unit that generates depth distribution information based on the parallax amount. 13. An image input unit for inputting first image data of a subject image obtained from a lens system having first and second viewpoints, and said two subject images formed on a light receiving plane. The first image data of the subject image obtained from the lens system having the image height characteristic in which the amount of parallax that occurs in the image is nonlinear with respect to the distance from the shooting position to the subject cancels the non-linearity of the amount of parallax. And an image processing unit for converting the image data into second image data. 14. The image processing unit according to claim 1, wherein the first image data is converted into the second image data in which the amount of parallax generated between the two subject images is proportional to a distance from a shooting position to the subject. The parallax amount correcting apparatus according to claim 13, wherein: 15. The image processing unit converts the first image data into two subject images when a subject on an equidistant plane from a shooting position is imaged through first and second viewpoints. 15. The parallax correction device according to claim 13, wherein the generated parallax is converted into the second image data such that the generated parallax is equal. 16. The image processing unit sets an azimuth between a normal passing through the shooting position and a line connecting an object on the equidistant plane from the shooting position and the shooting position, θ, and an image of the lens system. When the high characteristic is g (θ), the amount of parallax generated between the two subject images in the first image data is expressed by the following equation. The parallax amount correction apparatus according to claim 13, wherein the first image data is converted into the second image data by correcting the parallax with a correction function h that satisfies the following. 17. The apparatus according to claim 13, further comprising a depth distribution information generation unit configured to generate depth distribution information based on the amount of parallax.