JPH11234687A - High resolution digital camera and image information forming system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image with a high resolution by an image sensor with a small pixel number by regularly arranging primary color pixels of the same kind in a way that they are not adjacent to each other in the image sensor. SOLUTION: The center of each primary color pixel, red(R), green(G) and blue(B) is placed at each apex of a regular triangle in such a way that the primary color pixels of the same kind are not arranged at adjacent positions. For example, taking notice of a pixel B31, centers of the pixels B31, G32, R33 configure a regular triangle. A digital camera using this image sensor handles a basic unit of color expression to be the same as that of the primary color pixel. For example, taking the pixel number 31 as its unit, the blue B is an output of the primary color pixel whose number is 31 and the intensity of the other primary colors interpolates the output of the primary color pixels surrounding the pixel 31. Interpolation at an optional point on the inside of the triangle is easily calculated because of linear interpolation, and in the case that the triangle is a regular triangle, the calculation is more easily attained because the mean value of the intensity of three points only has to be calculated.

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital camera for converting a subject image into an electric signal and storing the same in a memory.
In particular, an increase in the number of pixels of the image sensor
Alternatively, the present invention relates to a digital camera and an image information forming system capable of achieving high resolution without requiring an expensive image forming optical system. 2. Description of the Related Art Digital cameras that take in an image an image formed by an optical system into a memory as an electrical signal by an image sensor have begun to spread, but in order to obtain a high-resolution image, a high-resolution image is required. It is necessary to employ an image optical system and an image sensor having a large number of pixels, and a high-resolution digital camera is inevitably an expensive product. Accordingly, it is an object of the present invention to realize and provide a high-resolution digital camera and an image information forming method using an inexpensive imaging optical system and an image sensor having a small number of pixels. It is. According to the present invention, as a means for inexpensively realizing a high-resolution digital camera, a high-resolution image similar to an image sensor having a large number of pixels is used while using an image sensor having a small number of pixels. And eliminating various aberrations such as chromatic aberration while using a relatively simple and cheap imaging optical system. As the former means, the relationship between the pixels in the image sensor and one unit of color expression is changed from the conventional one. The image sensor is configured by two-dimensionally arranging a large number of microsensors that output electric changes according to the intensity of light. In the related art, a filter for transmitting light of the primary colors is arranged in each of these microsensors to detect the light intensity of each primary color, and information of one unit of a color image is obtained by combining one set of the primary colors. Therefore, if the minute sensor that outputs in accordance with the light intensity of each primary color is a primary color pixel, if the unit of a color image is three primary colors, it becomes three.
It has an area of one primary color pixel. It is important to note that the information of these three primary color pixels is discarded even though they are slightly displaced from each other. In the present invention, a high resolution is obtained by positively utilizing the information based on the difference in the positions of the primary color pixels which have conventionally constituted the unit of the color image. That is, in the image sensor, the same type of primary color pixels are regularly arranged so as not to be adjacent to each other, and a pixel size of each primary color constitutes one unit of color. In addition to the output of the primary color pixel at that position, the intensity data of other primary colors is obtained by interpolation from the output intensities of three or more similar primary color pixels surrounding the pixel. By realizing a color image unit of the primary color pixel size in this way, without discarding information due to the difference in the position of each primary color pixel, it is possible to actively use and obtain a high resolution image with an image sensor with a small number of pixels. I can do it. Further, as a means for removing the latter various aberrations, a calibration mode is provided in the digital camera, and at the time of calibration, a predetermined calibration graphic image is taken in under predetermined conditions. Then, this image is compared with an ideal graphic image that should exist for the calibration graphic, and a mapping function for each pixel is stored in a memory in the camera as image correction information. The image is corrected based on the correction information each time a normal image is captured or afterwards. In this way, image distortion due to imperfections in the imaging optical system can be corrected, and this correction information is generated for each primary color of the pixel of the image sensor, and chromatic aberration is eliminated if correction is performed for each primary color. I can do things. In the present invention, the image is corrected pixel by pixel based on the correction information for each camera after photographing by partially utilizing the function of the control unit built in the digital camera. Various aberrations of the image forming optical system are effectively removed, and even if an image forming optical system including a cheap lens is used, image quality deterioration is suppressed, and a high-resolution image is realized. In addition, the adjustment of the imaging optical system may deviate over time or due to an accident, etc. after calibration. However, by incorporating a calibration program and preparing a calibration pattern, the digital camera can be calibrated appropriately and Correction information can be collected and updated, and high resolution and high quality can be maintained. A known calibration graphic is captured as an image, and correction information for each pixel is generated and stored to correct the captured image to correct various aberrations of the imaging optical system. By changing the configuration of the pixels and devising the unit of the color image information, the resolution of the color image can be increased by using the size of the primary color pixel as the unit of the color image. Embodiments of the present invention will be described below in detail with reference to the drawings. First, FIG. 1 shows functional blocks of a digital camera commonly referred to in each embodiment. In FIG. 1, an image of a subject is formed on an image sensor 12 by a lens 11 for calibrating an image forming optical system, and is activated by pressing a shutter button 18. And store it in the memory 14. Reference numeral 15 denotes a liquid crystal display, reference numeral 17 denotes a battery, reference numeral 16 denotes a power switch, and reference numeral 19 denotes a mode selector. An example in which high resolution is realized with a small number of pixels will be described as a first embodiment of the present invention. FIG. 2 shows a relationship between a primary color pixel and a unit of color expression of a conventional image sensor. The image sensor is provided with a filter on the surface of the minute sensor indicated by number 22, 23, 24, etc., which can transmit only light of primary colors, for example, red (R), green (G), blue (B), and the like. Obtain the electric output of the sensor to know the intensity of each color component of red, green and blue. The intensity information of these three primary colors is used as a unit of color expression, and a combination of three primary color pixels indicated by number 21 is used as a unit of a color image. In this case, red in the entire area of the number 21 is the number 22
, The green is the output of number 23 and the blue is the number 24
Output. Strictly speaking, at the position of number 24, the intensity of red should have already changed but is ignored. FIG. 3 shows an arrangement of primary color pixels in an image sensor used in a high-resolution digital camera according to a first embodiment of the present invention. It is assumed that the minimum unit primary color pixels have the same size in FIGS. In the figure, the centers of the three different primary color pixels of red (R), green (G), and blue (B) are arranged at the vertices of an equilateral triangle, and the same type of primary color pixel does not come to an adjacent position. Are arranged. For example, focusing on the primary color pixel B of No. 31,
G, R of number 33 and the center constitute an equilateral triangle, and numbers 31, 34, 35, or numbers 35, 36, 31 and the like each constitute an equilateral triangle. Using this image sensor, in the digital camera of the first embodiment of the present invention, the basic unit of color expression is considered to be the same size as the primary color pixels. For example, when the number 31 is considered as the unit, blue (B) is the output of the primary color pixel of the number 31,
The intensity of the other primary colors at that position is obtained by interpolating the output of the primary color pixels surrounding number 31. That is, the intensity of green (G) is obtained from the outputs of Nos. 32, 35 and 37, and the intensity of red (R) is obtained from No. 3
Interpolated from the outputs of 3, 34 and 36. The interpolation at an arbitrary point inside the triangle is a linear interpolation, so that the calculation is easy. In the case of a regular triangle, the intensity is the average of the three points and can be calculated more easily. This calculation is performed when the control unit 13 stores the output of the image sensor 12 in the memory 14.
Alternatively, the data may be stored in the memory 14 as it is, and may be subjected to a calculation and a process when the image is displayed on a display device having a high resolution. The latter will have less memory and will be more economical.
As described above, color information can be represented by the size of a unit of a conventional color image in one-third of a conventional color pixel.
Of course, it is not a simple matter that the resolution is tripled compared with the conventional one because of the interpolation effect, but high resolution is realized by effectively using the position information of at least one primary color pixel without discarding it. I can do it. An example in which various aberrations of the image forming optical system according to the second embodiment of the present invention can be corrected will be described with reference to FIG. FIG. 1 is used as a functional block diagram of the digital camera. In the calibration mode designated by the mode selector 19, a predetermined calibration graphic is set as a subject, and various conditions such as a distance or brightness between the calibration graphic and the lens 11 and further, an aperture, a shutter speed, and the like are set to predetermined conditions. While the image is captured below, the correct image that should be present for the graphic for calibration is stored separately in the memory 14 in the digital camera. The mapping relationship between them is derived as correction information. In other words, it is determined where the individual pixels forming the captured image should be located correctly, and the relationship indicating the position is used as correction information. At the time of normal photographing, the correct information is obtained by mapping and moving the pixels of the captured image using the correction information. At the time of normal photographing, the memory 1 is read from the image captured by the lens 11 and the image sensor 12.
The image data is converted and corrected for each pixel based on the correction information stored in No. 4 to complete the image. The derivation of the correction information and the correction of the image using the correction information will be described with reference to FIG. In FIG. 4, reference numeral 41 designates a part of a pixel in the image sensor 12, an image 42 of a calibration graphic taken in the memory 14, and a calibration graphic stored in the memory 14. An ideal image 43 for the figure is shown as being placed on the image sensor 12. Of course, it is necessary to strictly adjust the position, size, etc. in the image, so a plurality of points such as grid points are determined in advance as representative points, and the position, size, etc. are adjusted so that they match. This is a later image. The image 42 of the calibration graphic is distorted by the imaging optical system, and the straight line is curved when compared with the ideal image 43. This is exaggerated, but there may be other cases where it is partially distorted. The correction information in the present invention is information indicating how to correctly read the pixels of an image obtained by comparing these two images 42 and 43 to clarify the relationship between the pixels. For example, as shown in FIG. 4 (a), the pixels are numbered in column i and row j, and the luminance of the pixel (i, j) is indicated by A (i, j), and the pixel in FIG. In FIG. 4A, the pixel (i, j) in FIG. 4A is represented by (p, q), and the luminance at that point is represented by B (p, q). You only need to show the relationship of moving. Specifically, there are various methods, but it is possible to track where major points on the calibration figure are displaced, and to interpolate others from those relationships. However, as can be seen by comparing FIGS. 4 (a) and 4 (b), there are some pixels that have no destination in FIG. 4 (a), and there are also blank pixels which are not specified from anywhere in FIG. 4 (b). appear. Although the former is acceptable, the latter is not preferable because the image has a blank portion, and the brightness of those points is obtained by interpolation from surrounding pixels. B in this sense
(P, q) is a function of A (i, j). This is the correction information. Although the description is omitted in this embodiment, a method of obtaining the correction information in the spatial frequency domain is also conceivable. Although the amount of subsequent processing increases, the imaging optical system can be made closer to ideal, and the resolution can be further improved. As a third embodiment, an example in which the present invention is applied to an ultra-small card camera having a thickness of about several millimeters will be described. First, structural features will be described with reference to FIGS. explain. FIG. 5 is a perspective view of the card camera 50 during storage. In the figure, reference numeral 51 denotes a first plate, and reference numeral 52 denotes a second plate. Both sides are rotatably joined to each other by a joining member 53 made of a flexible resin. Numeral 55 denotes a display composed of liquid crystal, which is used as a finder for photographing to display an image captured by a CCD image sensor to be described later, or to display an image stored in a memory after photographing. used. Numeral 54 designates a key for releasing the latch for opening the first plate 51 and the second plate 52 at the time of use. When this key is pressed, the first and second plates 51 and 52 are set at a predetermined angle by a spring. open. When the first and second plates 51 and 52 are opened, the contacts constituting the necessary power switch are closed and the power is turned on. Thereafter, the key 54 also functions as a shutter switch. FIG. 6 shows the arrangement of main parts when the cross section of the card camera 50 is viewed from the direction indicated by reference numeral 56 in FIG.
In the drawing, one end 62 of a frame 68 supporting the lens 61 is rotatably fixed to the first plate 51, and the other end 63 is a sliding slot 64 provided in the second plate 52. Are rotatably and slidably fastened. Numeral 65 indicates a spring between the first plate 51 and the second plate 52, which always acts in a direction to push the first plate 51 and the second plate 52 apart. Therefore, when the key 54 in FIG. 5 is pressed, the latch between the first plate 51 and the second plate 52 is released, so that the first and second plates 51, 5 are acted by the action of the spring 65.
2 is pushed open, and at the same time, the end 63 of the lens 61 moves along the sliding long hole 64, and the lens 61 is raised. Reference numeral 66 denotes a light reflecting surface, and reference numeral 67 denotes an image sensor. FIG. 7 is a perspective view of the card camera 50 in a state where the first and second plates 51 and 52 are opened and an image can be taken. In the figure, the lens 61 is raised together with the frame 68 and positioned between the first and second plates 51 and 52,
A curtain 71 made of a flexible resin material and folded is stretched between the first and second plates 51 and 52 to constitute a dark room. FIG. 8 shows a cross section when the first and second plates 51 and 52 are opened as shown in FIG. 7, paying particular attention to the optical system. The light beam 82 reflected from the external object 81 is converged through the lens 61, is reflected by the reflection surface 66 fixed inside the first plate 51 on the way, and is fixed to the second plate 52. Incident on the image sensor 67. At that time, the object 81 is input to the image sensor 67 as an image 83, and the image 83 is converted into an electric signal by an instruction from a separate control unit and stored in a memory. The structure of the card camera 50 having an extremely thin structure has been described with reference to FIGS.
The image sensor described in the first embodiment of the present invention is used for the image sensor 67 of the present invention, and a built-in control unit (not shown) controls each pixel of the image sensor as described in the first embodiment. Edit the output to complete the high-resolution image. Although the reflecting plate 66 in FIG. 8 functions sufficiently even on a flat surface, it functions as a curved surface to share a part of the function of the lens 61. Furthermore, the control unit stores a calibration program and an ideal image corresponding to the calibration graphic, and corrects the image of the image sensor for each primary color pixel output as described in the second embodiment to remove chromatic aberration. In addition, digital processing complements the functions of the insufficient imaging optical system. Therefore, by using a lens system having such a thin structure but having a simple and insufficient chromatic aberration correction and performing digital processing based on the corrected correction information, it is possible to obtain a usable image. As described above, the embodiments of the present invention have been described in detail with reference to the drawings. An image sensor composed of two-dimensionally arranged primary color pixels such that primary color pixels of the same type are not adjacent to each other is used. An image information forming method for forming one unit of a color image by using the size of a primary color pixel, and generating and retaining correction information for correcting an image forming portion including an image sensor from an imaging optical system of a digital camera for each camera. Thus, a high-resolution and high-quality digital camera can be realized and provided without increasing the cost by a method capable of obtaining a high-quality image even with an inexpensive imaging optical system. However, the above description is for one embodiment of the present invention, and the above description does not limit the scope of the present invention at all, and various applications and modifications are made in accordance with the gist of the present invention. Is of course possible. For example, according to the present invention, a high-resolution image can be obtained with an image sensor having a small number of pixels, and a high-quality image with little distortion can be obtained with a cheap lens system. In order to obtain these characteristics, there are many processings by digital processing in a control unit in the camera. Naturally, these processings can be performed by reading necessary information together with an image into a personal computer outside the camera, for example. Although the embodiment does not particularly mention an example in which these processes are separated and independent from the camera, it is apparent that these processing programs are also included in the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a functional block diagram of a digital camera common to embodiments. FIG. 2 is a primary color pixel array of a conventional image sensor. FIG. 3 is a primary color pixel of an image sensor according to an embodiment of the present invention. Arrangement [FIG. 4] Image of calibration figure and desirable image [FIG. 5] Card camera 5 according to a third embodiment of the present invention
FIG. 6 is a perspective view of the card camera 50 according to the third embodiment. FIG. 7 is a perspective view of the card camera 50 in a photographable state. FIG. Sectional view for explaining the optical system when 51 and 52 are opened [Description of References] 11 ... Lens, 12 ...
Image sensor, 13 ... control unit,
14 ... memory, 15 ... liquid crystal display, 16 ... power switch, 17 ...
・ Battery, 18 ・ ・ ・ Shutter button, 19 ・ ・ ・ Mode selector 21 ・ ・ ・ Color image unit, 22 ・ ・ ・
Red primary color pixel, 23 ... green,
24 ... blue 31 ... primary color pixel R, 32 ...
Primary color pixels G, 33 ... primary color pixels B, 34, 35, 36
..Primary color pixels 41 ... pixels of image sensor, 42 ...
Calibration graphic image, 43 ... Ideal image 50 ... Card camera, 51 ...
A first plate, 52... A second plate,
53 ... joining member, 54 ... key,
55 ... display, 56 ...
Side 61 ... Lens, 62 ...
One end of the frame, 63 ... one end of the frame,
64: Slot for sliding, 65: Spring,
66: Light reflecting surface, 67: CCD
Image sensor 68, Lens frame 71 Curtain 81 External object, 82
Ray, 83 ... image

Claims (1)

1. An image sensor in which two primary-color pixels of the same type are regularly arranged two-dimensionally so as not to be adjacent to each other.
2. A high-resolution digital camera comprising an imaging optical system, a control unit, a memory, and the like. 2. A high-resolution digital camera according to claim 1, wherein color data is formed by three primary colors. 3. A high-resolution digital camera having a structure in which pixels in an image sensor are repeatedly arranged on the basis of a triangle having three primary color pixels as vertices. A high-resolution digital camera having a structure in which pixels in an image sensor are arranged on the basis of an equilateral triangle having three primary color pixels as vertices. In the high resolution digital camera according to the present invention, one unit of color expression is constituted by the pixel size of each primary color of the image sensor. An image information forming method characterized in that an output of a primary color different from the primary color of a corresponding pixel is interpolated from outputs of three or more same-color pixels surrounding the pixel to form image data. 2. A high-resolution digital camera according to claim 1, wherein said high-resolution digital camera is integrated with said image information forming system. 6. A high-resolution digital camera according to claim 5, wherein a predetermined calibration graphic is fetched as an image at the time of calibration, and an ideal image to be obtained by a built-in control unit or an external device added at the time of calibration. Is compared with the graphic image for calibration, correction information for each pixel is generated and stored in a memory in the camera, and the correction information is collected every time a normal image is taken or later. 7. A high-resolution digital camera and an image information forming method characterized in that an image is corrected based on the image sensor. A high-resolution digital camera and image information, wherein a calibration graphic is fetched for each primary color, correction information for each primary color is generated and stored in a memory, and an image is corrected for each primary color based on the correction information. Forming method [001]