JPS61191166A - Picture processing device - Google Patents

Abstract

PURPOSE:To convert a picture to a filtered electric signal in a short time by exposing the same picture on a solid-state image pickup element plural times while changing the relative position and taking out an induced accumulated electric charge in the image pickup element as an electric signal. CONSTITUTION:An input picture 2 is focused on a solid-state image pickup element 8 through a variable transmittance filter 15, a lens 3, and deflecting mirrors 4 and 5. Mirrors 4 and 5 are rotated by motors 6 and 7 driven by drivers 9 and 10 under the control of a controller CT 14 to shift the picture 2 on the element 8 in X and Y directions. By the control of the CT 14, the filter 15 exposes an image having the quantity of light proportional to a weight coefficient ai on the element 8 each time when the picture 2 is shifted on the element 8. When the number of picture elements in the range of weighted mean on the element 8 is defined as 3X3 and the brightness of each picture element is defined as Xi, a filtered output Y of a picture element X5 of the picture 1 is given by an equation. The electric charge stored on the element 8 by exposing the image on the element 8 plural times while shifting the position is taken out as the filtered electric signal, and this signal is amplified by an amplifier 13.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of the Invention] The present invention relates to an image processing device that converts an image into an electrical signal that has been subjected to filtering processing.

[Prior art]

Currently, there is a demand for image processing in a wide range of fields, and among these, filtering processing that removes the effects of noise and the like from image signals is even more important than basic processing. Filtering processing is performed, for example, when an image is divided into pixel columns and read, and the read value of a certain pixel is calculated as a weighted average of the outputs of a plurality of neighboring pixels including that pixel. Conventionally, in such processing, the output signal of the imaging device is multiplied by software using a computer or by dedicated software.

The addition was repeated. Therefore, when the number of pixels is large, the calculation time becomes extremely long), or expensive equipment has to be used to increase the processing speed.

[Summary of the invention]

An object of the present invention is to provide an image processing device with a simple configuration that converts an image into a filtered electrical signal in a short time.

4. The above-mentioned object of the present invention is to provide a solid-state image sensor, a means for exposing the same image to the solid-state image sensor multiple times while changing the relative position, and a means for exposing the same image to the solid-state image sensor multiple times while changing the relative position, and a method for removing charges induced by each exposure and accumulated in the solid-state image sensor. This is achieved by constructing an image processing device from means for extracting electrical signals.

〔Example〕

Hereinafter, the present invention will be described in detail with reference to the drawings.

FIG. 2 is an enlarged view showing a part of the image, and shows a 1-digit image. Filtering processing for image 1 is performed by the following calculation. Assuming that the range of pixels on which weighted averaging is performed is 3×3 pixels, the filtered output of pixel X5 is given by a Y-order equation, for example.

Y =, X aiXi (1) Here, Xi is the brightness of the pixel, and ai is a coefficient representing the weight for each pixel. Similar calculations are performed using the same weights for other pixels to perform filtering of the entire image.

FIG. 1 is a schematic configuration diagram schematically showing a first embodiment of an image processing apparatus of the present invention. In the figure, 2 is a manuscript, a film,
It is an input image such as photographic paper, 3 is a lens, 4 is a first mirror that deflects the light beam transmitted through the lens 3, and 5 is a mirror that deflects the light beam reflected by the first mirror 4 in a direction perpendicular to the melting direction. This is a useless second mirror. 6 and 7 are mirrors 4 and 7 respectively. As the motor for driving the mirror 5, an Ashi-Hulse motor is suitable. 8 is an image sensor (solid-state image sensor) placed at the imaging position of the lens 3;
rgeCoupleddevice) and B B D
(Bucked Brigade Device) etc. are used. Reference numerals 9 and 10 denote a motor driver for driving the motor 6 and the motor 7, and an image sensor driver for driving the 11 image sensors 8, which are controlled by a controller 14. Further, 12 is an electric signal taken out from the image sensor 8, and 13 is an amplifier that amplifies the electric signal 12. 15Fi is a variable transmission main filter generally used for spatial light modulators and the like, and its transmittance is varied in accordance with the signal from the controller 14.

An input image 2 is formed by a lens 3 onto an image sensor 8 via mirrors 4 and 5. The mirror 4 is rotated by the motor 6, and as a result, the image of the input image 2 on the image sensor 8 is shifted in the X direction. Similarly, the mirror 5 is rotated by the motor 7, and the height of the input image 2 is shifted in the X direction.Next, the principle of filtering according to the method of the present invention will be explained using FIG. Here, for the sake of simplicity, explanation will be given in terms of − dimension. In FIG. 3, 20 is an image of the input image 2 formed on the image sensor 8. In FIG. Further, 21 is an image of the input image 2 shifted by one pixel of the image sensor 8 in the X direction due to the rotation of the mirror 4, and 22 is an image of the input image 2 shifted by the same <2 pixels.

23, 24 and 25, 26 indicate the light receiving portions of the image sensor 8, respectively. Image 2 on light receiving section 23, 24, 25° 26
0 brightness as Xl, X2. Let it be X3°X4.

At this time, if the pixel range of the weighted average is 3 pixels, then
The filtering output Y2 of the image at the position corresponding to X2 is obtained as follows: Y2=, ΣaiXi (2). Here, ai is the weight for each pixel.

In FIG. 3, first, the aforementioned variable transmission main filter 15 is adjusted to expose an image 20 to the image sensor 8 with a light amount proportional to the weighting coefficient a1. At this time, the light receiving sections 23, 24 .
The charges accumulated in 25.26 are alXl and
Proportional to alX2, aIX3, and alX4. Next, by rotating the mirror 40, the image 20 is shifted by one pixel pitch and moved to the position of the image 21, and the transmittance of the variable transmission main filter 15 is changed to give the image 21 with a light amount proportional to a2 and charge accumulate. Since the charges are added to the initial charges and accumulated, the charges accumulated in the light receiving sections 23, 24, and 25 are respectively aIXl + a2X2. a1X
20a2X3, proportional to aIX3 +a2X4.

When the image 21 is further shifted by one pixel pitch by 40 rotations of the 1fC mirror and moved to the position of the image 22, and the image 22 is given with a light amount proportional to a3, the charges accumulated in the light receiving parts 23 and 24 are respectively a I Xl +a 2X2 +a
3X3.

It is proportional to aIX2+a2X3+a3X4. That is, the electric charge of the light receiving section 23 is the filtering output Y shown in equation (2).
It can be seen that it is proportional to 2. Therefore, if this charge is read out, a filtering output corresponding to X2 will be obtained. On the other hand, the filtering output ¥3 corresponding to X3 is Y3=, ΣaiXi+x (3)1, which is obtained from the amount of charge accumulated in the light receiving section 24. In this way, the entire image can be filtered by three exposures and two image shifts.

In FIG. 1, five mirrors are rotated by a motor 7 to shift an input image 2 on an image sensor 8 in the X direction. Therefore, by performing the same operation as in FIG. 3, two-dimensional filtering is possible. The principle of such two-dimensional filtering will be explained with reference to FIG. FIG. 4 is a schematic plan view of the image sensor 8, showing images of 30 input images on the image sensor.

31.32, 33, 34, 35, 36, 37°38.3
9 are light receiving parts of the image sensor 8, and the brightness of each part of the corresponding image 30 is expressed as xl, x2, . x3
．． x4. x5. x6. xl, x8. Let it be x9. Here, two-dimensional filtering is performed by sequentially shifting the images 30 and capturing the images with the image sensor 8.

FIG. 5 is a schematic diagram showing the order of image shifting in FIG. 4. 40 is a movement curve of the image 30 on the image sensor 8. A single arrow between the circles represents movement by one pixel pitch. When the image 30 moves on the image sensor 8 along the movement curve 40, the light receiving section 31 receives Xl,
X2゜x3. x, ,x5. x4. xl, x8. Each portion of the image 30 is exposed in the order x9. Therefore, the variable transmittance filter 15 is controlled to give an image 30 to the image sensor 8.
The light intensity of al, al, a3. a6. a5. a4. a
1, a8° and a9, the charge accumulated in the light receiving section 31 is proportional to ΣaIXi. This is the processing result of weighted average within 3 x 3 pixels for 1 = 1 for X5
□. This kind of charge accumulation is performed for all the light receiving parts of the image sensor 8 at the same time, so nine exposures and eight image shifts result in 3×
Filtering using 3 masks is performed over the entire image.

The two-dimensional filtering described above is executed in the apparatus shown in FIG. 1 as follows. The necessary control signals are provided by controller 14. First, the image sensor driver 11 sweeps out the charge from the image sensor 8 in response to a signal from the controller 14, and exposure is started. At this time, the controller 14 controls the variable transmission base filter 15 to transmit an amount of light proportional to al.

After exposing the image sensor 8 to the transmitted light, the controller 14 sends a signal to the motor driver 9, and the motor 6 shifts the mirror 4 so that the image of the input image 2 is shifted by one pixel pitch in the X direction on the image sensor 8. Rotate it as shown.

At the same time, the controller 14 switches the transmission group of the variable transmittance filter 15 and provides the image sensor 8 with an amount of light proportional to al. Further, the mirror 4 shifts the image in the y direction by one more pixel pitch in accordance with the signal from the controller 14, and exposes the image sensor 8 with a light amount proportional to a3 by similar light amount control.

Next, the controller 14 sends a signal to the motor driver 10, and the motor 7 rotates the mirror 5 in accordance with the drive signal from the motor driver 10 so that the image of the input image 2 is shifted by one pixel pitch in the X direction. After applying the light intensity proportional to a6 at this position, the controller 14 again sends a signal to the motor driver 9, and the motor 6 receives the input image 2.
The mirror 4 is rotated so that the image is shifted by one pixel pitch in the X direction. Here, the image sensor 8 has s a
An image is given with an amount of light proportional to 5. Mirror 4. When the image is shifted in the X and X directions by rotation of the mirror 5 and exposure is repeated, the two-dimensionally filtered image signal of the input image 2 is accumulated in the light receiving section of the image sensor 8. After the entire exposure is completed, the controller 14 gives a readout start signal to the image sensor driver 11, and the image signal 12 is read out from the image sensor 8. The image signal 12 is amplified by an amplifier 13 and sent to an appropriate processing circuit at the next stage.

Other embodiments of the present invention will be described below. In the drawing,
The same members as in FIG. 1 are given the same reference numerals, and detailed explanations are omitted.

FIG. 6 is a perspective view showing a second embodiment of the invention. 4
Parallel flat plates 1 and 42 are used to shift the input image 2 in the X direction and in the X direction, respectively, and are made of a transparent material such as glass or plastic. 43 and 44 are motors for driving the parallel plates 41 and 42, respectively.

The motor driver, image sensor driver, variable transmittance filter, and controller are the same as those shown in the first embodiment, so they are not shown. In this second embodiment, the image shift of the input image 2 is carried out on a parallel plate 4.
Perform by 1.42 rotations □.

FIG. 7 is a front view showing the principle of image shift due to rotation of parallel plates. Here, 45 is an imaging light flux that comes out from the 201 points of the input image and forms an image on the image sensor 8.
is the image point. Further, 47 indicates a state in which the parallel plate 41 is rotated.

48 is an imaging light beam for the rotated parallel plate 47;
This is the image point of the 49-image-forming light beam 48.

On the image sensor 8, the imaging light beam 45 from a certain point in the input image 2 when passing through the parallel plate 41 is at the image point 4.
Attach the statue to 6. On the other hand, with respect to the parallel plate 47 which has rotated once, the imaging light beam 48 naturally passes through the same optical path as the imaging light beam 45 until it is incident on the parallel plate 47, but forms an image at an image point 49 after entering the parallel plate 47. Image point 49 is shifted in the X direction with respect to image point 46. Now, the parallel plate 41 has been rotated about an axis perpendicular to the plane of the paper, but if it is rotated about an axis parallel to the plane of the paper, the image point will shift in the X direction.

In FIG. 6, a parallel plate 41 is rotated by a motor 43 to shift the input image 2 on the image sensor 8 in the X direction. On the other hand, the parallel plate 42 is rotated by a motor 44 to shift the image in the X direction. Filtering in the second embodiment is performed in exactly the same manner as in the first embodiment by repeating the image shift and the exposure process in which an amount of light proportional to the weighting coefficient ai is applied to the image sensor 8.

FIG. 8 is a perspective view showing a third embodiment of the present invention.

In this embodiment as well, only the self-shifting means is illustrated as in FIG. 6. 501 and 502 are glass plates, and 51 is a transparent elastic body sandwiched between the glass plates 501 and 502. A suitable material for the transparent elastic body 51 is, for example, silicone rubber 36 (trade name: KE104Ge/, manufactured by Shin-Etsu Chemical), and adhesion to the glass plates 50□, 502 can be achieved using a silane coupling agent. Glass plate 501
502 is parallel to the y-axis and forms a certain angle to each other to form a prism together with the transparent elastic body 51. 521
e 522 is a glass plate, and 53 is the same transparent elastic body as 51. The glass plates 521 and 522 are parallel to the X axis, and together with the transparent elastic body 53 form a prism.

The transparent elastic body 51.53 is a glass plate 501 r 502,
Alternatively, the prism can be a variable apex angle prism so that when s21.522 is moved, the shape changes accordingly. It is well known that when a prism is inserted into the optical path of an imaging optical system, the light beam will deflect and the image will shift according to its apex angle, and as the prism apex angle changes, the image will shift continuously. do.

The prism formed by the glass plates 501 and 502 and the transparent elastic body 51 changes its apex angle by moving the glass plate 502 by a drive system (not shown), and changes the image of the input image 2 on the image sensor 8 in the X direction. shift. on the other hand,
The prism formed by the glass plates 521 and 522 and the transparent elastic body 53 changes its apex angle by moving the glass plate 522 by a drive system (not shown), thereby shifting the image of the input @*2 in the X direction.

Filtering in the third embodiment is also performed by repeating the process of image shifting and exposure for applying a light amount proportional to the weighting coefficient al to the image sensor 8, just as in the first and second embodiments.

FIG. 9 is a schematic cross-sectional view showing another example of the configuration of the variable apex angle prism. 60 indicates a plano-concave lens, 61 indicates a plano-convex lens,
The curvature of the concave surface of the plano-concave lens 60 and the curvature of the convex surface of the plano-convex lens 61 differ only in sign but have the same absolute value. When the concave surface of the plano-concave lens 60 and the convex surface of the plano-convex lens 61 are aligned and the plano-convex lens 61 is slightly shifted, the apex angle Φ is obtained as shown in FIG. 9(a).
When the plano-convex lens 61 is further shifted, it functions as a prism with an apex angle of Φ2 as shown in FIG. 9(b). Even with such a configuration, a variable apex angle prism can be realized. Therefore, even if this prism is inserted into the imaging optical path shown in the third embodiment instead of the variable apex angle prism using a transparent elastic body, filtering can be performed in the same way.

FIG. 10 is a schematic sectional view showing only the image shifting means of the fourth embodiment of the present invention. 7o1.7o2 is an electro-optical material, B 80 , LiNbO3, K DP .

PLZT or the like is used. 711.712.713
.

714 is a transparent electrode, and ITO or the like is suitable. 7
3 is a birefringent material such as calcite, and 74 is also a birefringent material, but its thickness is half that of the birefringent material 73. 72 is the incident light, and 75 and 76 are the incident light 7, respectively.
These are ordinary rays and extraordinary rays in the birefringent material 73 corresponding to 2. 77.78 are ordinary rays 75. These are the ordinary rays and extraordinary rays in the birefringent index material 74 corresponding to 79
, 8.0 are the ordinary ray and extraordinary ray in the birefringent base material 74 corresponding to the extraordinary ray 76, respectively. 81.82,83
．． 84 are the ordinary ray 77 and the extraordinary ray 78 . Ordinary ray 7
9. This is the outgoing light of the extraordinary ray 80.

Transparent electrodes 711 and 712 are provided on both sides of the electro-optic material 701.
is provided, and by applying a voltage of about IKV to this, the electro-optic material 701 functions as a polarization rotation element. That is, the incident light 72 is linearly polarized, and the polarization direction is inclined by 45° with respect to the two main axes of the electro-optic material 701. When the voltages applied to the transparent electrodes 711 and 712 are appropriately controlled, the light emitted from the electro-optic material 701 becomes linearly polarized light that is the same as the incident light 72 or linearly polarized in a polarization direction perpendicular thereto. The light emitted from the electro-optic material 701 becomes the ordinary ray 75 or the extraordinary ray 76 of the birefringent material 73 depending on its polarization direction.
When refracted at the surface of the ray 75, it does not follow the usual Snell's law and travels in a direction different from that of the ordinary ray 75. Therefore, when emitted from the birefringent material 73, the ordinary ray 75. extraordinary ray 7
6 are emitted from different positions.

The electro-optic material 702 is provided with a transparent electrode "a=714," and the direction of polarization of the incident linearly polarized light is rotated by 90 degrees by applying a voltage.The polarization direction is changed to linearly polarized light.

The polarization direction of the ordinary ray 75 is controlled by the electro-optic material 702 and becomes an ordinary ray 77 or an extraordinary ray 78 in the birefringent material 74 . Similar to the propagation in the birefringent material 73, the ordinary ray 77 and the extraordinary ray 78 travel in different directions in the birefringent material 74 and exit from different positions. However, birefringent material 7
4 is half the thickness of the birefringent material 73, so the separation distance between the ordinary ray 77 and the extraordinary ray 78 at the time of emission is equal to the thickness of the birefringent material 73.
This is half of the case. Similarly, the polarization direction of the extraordinary ray 76 is controlled by the electro-optic material 702, and it becomes an ordinary ray 79 or an extraordinary ray 80 in the birefringence material 74. The separation distance between the ordinary ray 79 and the extraordinary ray 80 upon emission is the same as the separation distance between the ordinary ray 77 and the extraordinary ray 78.

The outgoing lights 81, 82, 83, 84 are shifted by the same distance from each other, and the outgoing lights 81, 82, 83, 84 are shifted by the same distance.
Which of 4 is selected depends on the combination of voltages applied to the electro-optic materials 701 and 702, for example, as shown in the table below.

Since it is possible to shift the image with the element shown in Fig. 10, for example, when performing filtering with a 3 x 3 mask, the output light 81, 82, 83 is used to shift the image in the X direction, If another set of elements is arranged rotated by 90 degrees and used as a partner shift element in the X direction, the same processing as in the first embodiment can be performed.

In the above explanation, it has been described that the weighted average is executed based on the pixel information of the image of the input image 2 corresponding to the position of the light receiving part of the image sensor 8, but in reality, it is not limited to this and the value of a pixel at an arbitrary position is executed. Can be used.

FIG. 11 is a plan view showing the image and the positions of pixels used for weighted averaging. 901. . . . , 909 are light receiving sections of the image sensors 8, which are arranged in a row separated from each other by Ka. 91□,...,91. are pixels located in the image 30 at a distance b from each other.

In the above description, it has been described that the image 30 is shifted by an integer multiple of a, but an unfiltered output can be obtained even if the image is shifted by an arbitrary value of b that is different from the value of a. For example, for filtering using a 3×3 mask centered on the light receiving portion 905, the light receiving portion 905 and the pixel 911...
This can also be done using the image information of the image 30 at the position 918.

In the case of conventional electrical filtering processing, the pixel information that can be used is limited to that corresponding to the position of the light receiving part of the image sensor 8, but with the device of the present invention, pixel information can be brought from any position. The degree of freedom in calculation increases.

Furthermore, with the configuration of the device of the present invention, the resolution of a small number of image readings can be increased above the resolution determined by the number of pixels of the image sensor.

FIG. 12 is a plan view showing the positions of images and image reading pixels. 931. ......, 9316 is the light receiving section 9
01...This is the position of the pixel when the image is completely filled in the X and X directions with a pitch i that is half the pitch a of the light receiving part, with the position 909 as a reference.

First, images are input at the positions of the light receiving sections 90□, . . . 909 and stored in the memory. Next, move the image 30 in the X direction 1
.

2a software and input the image, pixels 931, 932
938, 939.93.5.9316 can be input and also stored in the memory. Further move the image 30 in the X direction by 2 a
If you shift and input the image, pixels 93λh 936+
9311.

9313 can be entered and stored in memory. Finally statue 30
If you manually shift the image by 1a in the X direction again, the pixels will be 933.935.937.9310,93□2゜93
14 can be input, and after storing this in the memory, combining the already stored pixel signals and arranging them at a predetermined position, the image will be twice as large in the X direction compared to inputting the image as is with the image sensor 8, I am amazed at the fact that the resolution has increased by 2 times in the direction and 4 times in total for input.

FIG. 13 is a diagram showing the results of filtering processing according to the method of the present invention, in which (a) is an image before processing, and (b) is an image after processing. Here, the horizontal axis shows the position, and the vertical axis shows the light intensity at that position. As described above, according to the present invention, a noisy original image as shown in 96 can be made into a processed image from which noise has been removed, as shown in 97. In the present invention, the number of pixels and weighting coefficients to be weighted average are determined depending on the degree of filtering desired, but in the simplest case, the pixel range is set to 3 x 3 pixels and nine weighting coefficients are set as shown in Fig. 4. All 1
You should do it. In order to obtain very good filter characteristics, there are various methods, such as expanding the pixel range to 5×5 pixels, etc., and determining the weighting coefficient according to a Gaussian function.

The present invention is not limited to the above-described embodiments, and can be applied in various ways. For example, in the embodiment, the exposure amount was adjusted using a variable transmittance filter, but it is also possible to adjust the exposure amount by providing a shutter between the input image and the image sensor and changing the opening time of this shutter according to the weighting coefficient. Furthermore, the brightness of the illumination light source of the input image may be changed.

Further, as a method of shifting, in addition to the method of moving the image and the image sensor as in the present invention.

A method of exposing to light while transferring charges can be considered, and the present invention can be used in combination with such a method. For example, by performing a shift in the X direction by charge transfer,
If the image is shifted in the direction as in the present invention, there is an advantage that both the optical image shifting mechanism and the structure of the image sensor do not need to be very complicated.

〔Effect of the invention〕

As explained above, according to the image processing device of the present invention,
Since arithmetic processing is performed on the image sensor, there is no need for a computer or special hardware to process the output electrical signals, making the device configuration simple and inexpensive. Also,
Since processing is performed on all pixels in parallel, the processing time remains the same even if the number of pixels increases, and even a large amount of image information can be processed at high speed.

[Brief explanation of drawings]

FIG. 1 is a schematic diagram showing a first embodiment of an image processing device according to the present invention, FIG. 2 is a plan view showing a part of an image, and FIG. 3 is a schematic cross-section of an image sensor illustrating the principle of filtering according to the present invention. 4 is a plan view of an image sensor showing the principle of two-dimensional filtering, FIG. 5 is a schematic diagram showing the order of image shifting in FIG. 4, and FIG.
7 is a schematic diagram showing the image shifting means of the embodiment; FIG. 7 is a schematic sectional view explaining the principle of image shifting in the apparatus of FIG. 6; FIG.
The figure is a schematic diagram showing an image shifting means according to a third embodiment of the present invention;
Figure 9(a). (b) is a schematic sectional view showing another configuration example of the variable apex angle prism in the apparatus shown in FIG. 8; FIG. 10 is a schematic sectional view showing the image shifting means of the fourth embodiment of the present invention; FIG. 12 is a plan view showing the positional relationship between pixels and light receiving sections in the device of the present invention, and FIGS. 13(a) and 13(b) are diagrams showing images before and after filtering processing, respectively. 2... Input image, 3... Lens, 4, 5... Mirror, 6.7... Motor, 8... Image sensor, 9
．． 10... Motor driver, 11... Image sensor driver, 12... Electric signal, 13... Amplifier,
14...Controller, 15...Variable transmission wheel filter. Figure q (phantom stop OU 61 f) U 577I
, ri/274 Z4 1st/Figure (-c1

Claims (1)

[Claims] (1) An image processing device that converts an image into an electrical signal, comprising: a solid-state image sensor; a means for exposing the image to the solid-state image sensor multiple times while changing its relative position; An image processing device comprising means for extracting charges accumulated within a solid-state image sensor as the electric signal.