JP2006105771A - Imaging device and topographical map preparing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem wherein a small apparatus cannot obtain stereoscopic information with a single imaging operation. <P>SOLUTION: There are provided with an imaging lens; an imaging element; an image capture means for independently taking images obtained from light fluxes through at least two different pupil areas of the imaging lens; a detection means for detecting distance distribution information on an object, from at least two of the images obtained from the image capture means; a separation means for separating the images for each distance to the object according to the distance distribution information, from the two images; and a compositing means for re-compositing a pair of stereoscopic images, having disparity different from that of the two pupil areas. Thus, the stereoscopic image, having desired disparity, can be obtained by one photographing. Further, there are provided an imaging position detection means for detecting the position, the height, and the direction of the imaging place and a map information generation means for generating map information, including contour lines from at least one of the images, the distance distribution information, and the imaging position information. Thus, an actual photographed topographical map with the contour lines can be prepared. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging device and a topographic map creation device that can obtain information for a stereoscopic image based on subject distance information.

  Conventionally, many proposals have been made regarding stereoscopic image cameras.

  In the figure recognition device disclosed in Japanese Patent Laid-Open No. 50-23740, a subject is photographed with two cameras and a reproduced image is obtained in which the parallax between the left eye and the right eye is shifted left and right. It is.

  The stereoscopic image display device disclosed in Japanese Patent Publication No. 55-36240 is a stereoscopic image display device that obtains stereoscopic image information by combining flat screen information and depth information. It is obtained from the correlation of image information taken with a camera, or obtained by using altitude measuring means such as ultrasonic waves or radio waves.

In the stereoscopic electronic still camera disclosed in Japanese Patent No. 3112485, the subject distance distribution is detected by scanning the focusing lens and detecting the distribution of the contrast peak.
Japanese Patent Laid-Open No. 50-23740 Japanese Patent Publication No.55-36240 Japanese Patent No. 3112485

  However, in the above conventional example, three-dimensional information could not be obtained by a single capture with a small device.

  In the conventional examples disclosed in Japanese Patent Laid-Open Nos. 50-23740 and 55-36240, it is inevitable that the apparatus for obtaining the three-dimensional information is increased in size and complexity.

  In the conventional example of Japanese Patent No. 3112485, accurate distance information cannot be obtained when the subject moves during the scanning time of the focusing lens.

  In order to solve the above-described problems, an imaging apparatus according to the present invention includes an image capturing unit that independently captures an image obtained from a light beam that has passed through a photographing lens, an imaging element, and at least two different pupil regions of the photographing lens. Detection means for detecting subject distance distribution information from at least two images obtained from the image capturing means, and separation for separating the images for each subject distance from the two images corresponding to the distance distribution information And combining means for recombining the two pupil regions with a pair of stereoscopic images having different parallaxes.

  By the means described above, the present invention can obtain a three-dimensional image by obtaining three-dimensional information by a single capture with a small device.

  In addition, parallax can be enlarged or reduced.

[First embodiment]
1 to 7 show an embodiment of the present invention, and FIG. 1 is a main block diagram of an imaging apparatus of the present invention.

  The imaging apparatus uses a two-dimensional imaging element capable of performing imaging and defocus detection simultaneously, captures a subject image and a defocus distribution at the same time, and generates three-dimensional image information based on the defocus distribution information.

  In FIG. 1, reference numeral 1 denotes an image pickup means, and the image pickup means 1 is configured in the same manner as a conventional image pickup device disclosed in Japanese Patent Publication No. 2002-14277, and can acquire a defocus distribution simultaneously with image pickup. It has become.

  Defocus detection means 2 outputs defocus distribution information for the entire screen.

  Defocus map storage means 3 divides the entire screen of the output of the defocus detection means 2 into predetermined areas, and calculates and stores representative defocus values for each area.

  An image processing unit 4 performs predetermined image processing such as color matrix processing and gamma processing from the output signal of the imaging unit 1, and stores the image data in the image temporary storage unit 5 as image data.

  Reference numeral 6 denotes a three-dimensional image generation means. The three-dimensional image generation means 6 converts the image data into a pair of image data having parallax based on the defocus distribution information stored in the defocus map storage means 3.

  The distance information shown in the defocus map is converted into parallax information when a predetermined observation system is assumed. The predetermined observation system is an observation system obtained at a distance between human eyes to obtain a natural stereoscopic effect, and an observation system having a larger interval to obtain a more emphasized stereoscopic effect. .

  When the distance between the two eyes of the observation system is determined, the parallax for each distance is obtained by calculation.

  When the parallax for each distance is obtained as a parallax map corresponding to the defocus map, the image data is reconstructed as a pair of data having a deviation based on the parallax information.

  At the time of reconstruction, it is necessary to perform separation for each distance of the image data. However, it is possible to collectively convert the distance for each certain distance area as a representative distance. When using the representative distance, it is preferable to recognize the subject image, extract the contour of the subject for each distance, and create an image with the same representative distance in the contour region.

  By performing this process, it is possible to avoid causing the image to feel remarkably strange even if an image distance measurement error occurs.

  At the time of reconstruction, an image that should appear from the shadow of the subject in the foreground does not have an actual parallax image, and is generated by calculation from surrounding images.

  7 is equidistance information generating means for generating equidistance diagram data connecting the equidistant positions of the defocus information of the defocus map.

  FIG. 2 is a configuration diagram of the imaging apparatus of the present invention.

  In the figure, reference numeral 10 denotes an imaging apparatus body, 11 denotes a photographing lens of the imaging apparatus, 12 denotes an optical low-pass filter, 13 denotes an infrared cut filter, and 14 denotes an imaging element. The image sensor 14 has the configuration shown in FIG. 3, and is composed of a solid-state image sensor.

  In FIG. 3, two photodiode arrays each having a color filter are paired and correspond to one microlens.

  In the above configuration, a set of photodiodes corresponding to the same side of each microlens having color filters of the same color as G111-G311 ..., G112-G312 ... By taking it, it is possible to obtain a defocus distribution. In this embodiment, a color filter is taken into consideration, and correlation is obtained for each of RGB.

  Of course, if the black and white type is acceptable, there is no need.

  The image signal obtained from this image sensor is output by separating light beams coming from different pupil positions of the lens as two images.

  Distance information can be obtained by correlating these two images.

  A timing generator (TG) 15 drives the image sensor 14.

  Reference numeral 16 denotes an A / D converter (A / D) which samples an analog signal output from the image sensor 14 and converts it into a digital signal.

  A primary buffer memory 17 temporarily stores output data of the A / D converter.

  The taking lens 11 to the primary buffer memory 17 correspond to the image pickup means 1 in FIG.

  A correlation operation circuit 18 corresponds to the defocus detection means 2 in FIG.

  The correlation calculation circuit 18 performs correlation between the two images stored in the primary buffer memory 17 on the entire screen, and determines the defocus amount for each portion of the screen.

  FIG. 4 is a characteristic diagram for explaining the two series of data. By taking the correlation of the two series of signals, the amount of deviation (Δ1, Δ2,.・ ・) Is obtained.

  The determined defocus amount is stored in the defocus buffer memory 19.

  Reference numeral 20 denotes an image preprocessing processor, and 22 denotes an image buffer memory.

  The image preprocessor 20 performs color matrix processing, gamma processing, noise reduction processing, and the like on the image data in the primary buffer memory 17 and stores them in the image buffer memory 22 as two images.

  Reference numeral 23 denotes a CPU which controls the entire camera, performs three-dimensional data processing, recording processing, and the like.

  Reference numeral 24 is an external interface, and 25 is an external storage device. The external storage device 25 is composed of a nonvolatile storage medium such as a flash memory.

  Reference numeral 26 denotes a release switch which is a photographing instruction means, 27 is a menu button, 28 is a cross key for selecting settings and the like, and 29 is a set button.

  Reference numeral 30 denotes a power switch for turning on / off the power of the imaging apparatus.

  Reference numeral 31 denotes a display panel as a display means, and the display panel 30 is constituted by a TFT color liquid crystal panel with a backlight.

  The display panel 31 is used for a finder for framing determination at the time of shooting, menu operations for checking a shot image, checking an equidistance map, and various settings of the imaging device.

  Next, the operation of the imaging apparatus will be described using the flowchart of FIG.

When the power switch 30 is turned on, the operation starts from S101.
(S101)
The image sensor 14 and the like are set in the finder mode, the signal of the image sensor 14 is taken out in real time, and an image is displayed on the display panel 31 as a finder image.
(S102)
When the power switch 30 is turned off, the operation of the device is terminated.
(S103)
If the release switch 26 is pressed, the process proceeds to S150.
(S104)
If the menu button 27 is pressed, the process proceeds to S110.

  If not, the process returns to S102.

The operation input of the device is monitored in S102 to S104.
(S110)
The image sensor 14 is turned off.
(S111)
A menu is displayed on the display panel 31.

  At this time, the default selection item of the menu is highlighted to indicate the selected content of the current menu.

The display form of the menu is shown in FIG.
(S112)
If the menu button 27 is pressed, the process proceeds to S101.
(S113)
If there is an input from the cross key 28 or the set button 29, the process proceeds to S120.
(S114)
When the power switch 30 is turned off, the operation of the device is terminated.

If there is no input, return to S112
(S120)
Perform menu setting processing.

  In the menu setting process, the following process is performed.

  When the up and down of the cross key 28 is pressed, the menu item is moved.

  When the left and right of the cross key 28 are pressed, the user enters and exits the hierarchical menu. Press right to move to the lower level, and press left to move to the upper level.

The set value of the item is determined by the set button 29.
(S121)
If the setting value has been changed, a predetermined setting change is performed.

  Thereafter, the process returns to S112.

From S150, a sequence of shooting and data creation is performed.
(S150)
The CPU 23 sets TG15 to the main shooting mode.
(S151)
The CPU 23 drives the image sensor 14 via the TG 15 to execute the main photographing.
(S152)
The output of the image sensor 14 is converted into digital data by the A / D 16 and stored in the primary buffer memory 17.
(S153)
The correlation calculation circuit 18 performs correlation between the two images stored in the primary buffer memory 17 on the entire screen, calculates a defocus amount for each portion of the screen, and stores it in the defocus buffer memory 19. This defocus amount corresponds to the subject distance.
(S154)
The image preprocessor 20 performs color matrix processing, gamma processing, noise reduction processing, and the like on the image data in the primary buffer memory 17 and stores them in the image buffer memory 22 as two images.
(S155)
The CPU 23 performs image separation of the subject on the image stored in the image buffer memory 22 while referring to the value of the defocus buffer memory 19.

  In the image separation, the defocus amount value is approximate and a portion where the contour of the subject can be recognized is separated as a single image and a representative defocus value is set.

  Also, there are originally two images, but at this point they are combined into one.

The separated image is stored in the image buffer memory 22 together with the representative defocus value.
(S156)
A pair of images are generated by shifting the separated image generated in S155 so as to have a predetermined parallax according to the subject distance obtained from the representative defocus value. This process is performed for all separated images.
(S157)
The image obtained in S156 is combined into two images.

At this time, an image of a region that is insufficient due to superposition is generated by interpolation from an image at the back.
(S158)
With reference to the value in the defocus buffer memory 19, an equal subject distance diagram when the subject distance is divided at a predetermined interval is created.
(S159)
Two images and an equal subject distance diagram are stored as a set in the external storage device 25 via the external interface 24.

At this time, a related file name may be used, a method of storing the data collectively as a single data, or a method of adding tag information.
In addition, the stored content varies depending on the setting of the imaging device.
(S160)
The image sensor 14 and the like are set in the finder mode, the signal of the image sensor 14 is taken out in real time, and an image is displayed on the display panel 31 as a finder image.

  Thereafter, the process returns to S102.

  Thus, the operation of the imaging apparatus is performed.

[Second Embodiment]
7 to 8 show a second embodiment of the present invention and show an example incorporated in a radio-controlled small aircraft.

  The same numbers are assigned to portions corresponding to those in the first embodiment.

  Reference numeral 50 denotes an imaging apparatus main body, which has the function of a wirelessly controlled small aircraft.

  The imaging device main body 50 includes a wireless transceiver 51 and can be controlled wirelessly.

  52 is an engine, and 53 is a steering device.

  55 is a GPS unit for grasping the position, 56 is an orientation detection unit for grasping the orientation of the device, 57 is an acceleration sensor for grasping the acceleration status of the device, and 58 is an altitude for grasping the altitude of the device. A sensor 59 is a ground speed sensor for grasping the speed of the device, and 60 is a tilt sensor for grasping the tilt of the device.

  The position and direction of the imaging device in the space are grasped with 55 to 60 sensors.

  61 is a controller.

  The controller 61 incorporates a transceiver 62 and communicates with the imaging device main body 50 wirelessly.

  Reference numeral 63 denotes an operation lever that controls the flight operation of the imaging apparatus main body 50.

  A display panel 64 displays an image sent from the imaging unit of the imaging apparatus main body 50.

  Reference numeral 65 denotes a photographing instruction switch.

  66 is a CPU and 67 is a buffer memory.

  In the present embodiment, the imaging apparatus main body 50 can be remotely operated to perform shooting / recording operations at any timing.

  In addition to the first embodiment, the shooting to recording operation simultaneously records position information at the time of shooting.

  Based on this position information, it is possible to create a topographic map having detailed real image information including height information on the ground using a captured stereoscopic image and an equidistant map.

  The photographing, data creation, and recording operations may be automatically performed every predetermined distance movement or every predetermined time.

  If configured in this way, a wide range of topographic map data can be easily acquired.

Main block diagram of imaging device Configuration diagram of imaging device Image sensor block diagram Characteristic chart of 2 series data Flow chart showing operation of imaging device Illustration of menu The block diagram of the imaging device of 2nd embodiment Controller configuration diagram

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging means 2 Defocus detection means 3 Defocus map storage means 4 Image processing means 5 Image temporary storage means 6 Three-dimensional image generation means 7 Equidistant information generation means

Claims (4)

  From an imaging lens, an image sensor, an image capturing unit that independently captures an image obtained from a light beam that has passed through at least two different pupil regions of the photographing lens, and at least two images obtained from the image capturing unit The detection means for detecting the distance distribution information of the subject, the separation means for separating the image for each distance of the subject corresponding to the distance distribution information from the two images, and the two pupil regions have different parallaxes An image pickup apparatus having a synthesizing unit that re-synthesizes a pair of stereoscopic images.   From an imaging lens, an image sensor, an image capturing unit that independently captures an image obtained from a light beam that has passed through at least two different pupil regions of the photographing lens, and at least two images obtained from the image capturing unit An image pickup apparatus, comprising: detection means for detecting distance distribution information of a subject; and an equidistance line indicating an equidistant region for each predetermined distance from the distance distribution information.   From an imaging lens, an image sensor, an image capturing unit that independently captures an image obtained from a light beam that has passed through at least two different pupil regions of the photographing lens, and at least two images obtained from the image capturing unit Detection means for detecting distance distribution information of a subject, imaging position detection means for detecting the position, height and direction of an imaging location, and generating map information including contour lines from at least one of the images, distance distribution information and imaging position information An apparatus for creating a topographic map, characterized by having a map information generating means.   4. The topographic map creating apparatus according to claim 3, further comprising a moving unit that spatially moves the imaging unit, and generating a plurality of map information as the imaging unit is moved by the moving unit. Topographic map creation device.

Images