JPH0547017B2 - - Google Patents

Description

[Detailed description of the invention]

[Field of the Invention] The present invention relates to an image processing device that converts an image into an electrical signal that has been subjected to filtering processing. [Prior Art] Currently, there is a demand for image processing in a wide range of fields.
Among these, filtering processing that removes the effects of noise and the like from image signals is important as a basic processing. For example, when reading an image by distributing it into pixel rows, filtering processing is used to divide the read value of a certain pixel into
This is performed by calculating the weighted average of the outputs of multiple pixels in the vicinity of the pixel.
Conventionally, in such processing, the output signal of the imaging device is repeatedly multiplied and added using software using a computer or dedicated hardware. Therefore, when the number of pixels is large, the calculation time becomes extremely long, or expensive equipment has to be used to increase the processing speed. [Summary of the Invention] An object of the present invention is to provide an image processing device with a simple configuration that converts an image into a filtered electrical signal in a short time. The above-mentioned objects of the present invention include a solid-state image sensor, a means for exposing the same image to the solid-state image sensor multiple times while changing the relative position, and a means for changing the amount of exposure light during the plurality of exposures. This is achieved by configuring an image processing apparatus from means for extracting the charges induced by the exposure and accumulated in the solid-state image sensor as electrical signals. [Example] The present invention will be described in detail below with reference to the drawings. Figure 2 is an enlarged view showing a part of the image, 1
indicates an image. Filtering processing for image 1 is performed by the following calculation. Assuming that the range of pixels on which weighted averaging is performed is 3×3 pixels, for example, the filtered output Y of pixel X ₅ is given by the following equation. Y= ₉ 〓 ⁱ⁼¹ a _i X _i (1) Here, X _i is the brightness of the pixel, and a _i is a coefficient representing the weight for each pixel. The same weights are used for other pixels and similar calculations are performed,
Filtering of the entire image is performed. FIG. 1 is a schematic configuration diagram schematically showing a first embodiment of an image processing apparatus of the present invention. In the figure, 2 is an input image such as a manuscript, film, photographic paper, etc., 3 is a lens, 4 is a first mirror that deflects the light beam transmitted through the lens 3, and 5 is the light beam reflected by the first mirror 4. is a second mirror for deflecting in a direction orthogonal to the above. 6 and 7 are respectively mirror 4,
This is a motor for driving the mirror 5, and a pulse motor is suitable. 8 is an image sensor (solid-state image sensor) placed at the imaging position of the lens 3;
CCD (Charge Coupled Device) and BBD
(Bucked Brigade Device) etc. are used.
Motor drivers 9 and 10 drive the motors 6 and 7, respectively, and an image sensor driver 11 drives the image sensor 8, which are controlled by a controller 14. Further, 12 is an electrical signal taken out from the image sensor 8, and 13 is an amplifier that amplifies the electrical signal 12. Reference numeral 15 denotes a variable transmittance filter that is generally used in spatial light modulators and the like, and varies its transmittance in accordance with signals from the controller 14. Input image 2 is focused by lens 3 onto image sensor 8 via mirrors 4 and 5 . The mirror 4 is rotated by the motor 6, so that the image of the input image 2 on the image sensor 8 is shifted in the x direction. Similarly, the mirror 5 is rotated by the motor 7,
The image of input image 2 is shifted in the y direction. Next, the principle of filtering according to the method of the present invention will be explained using FIG. Here, for the sake of simplicity, a one-dimensional explanation will be given. In FIG. 3, 20 is an image of the input image 2 formed on the image sensor 8. In FIG. Further, 21 is an image of the input image 2 that has been shifted by one pixel of the image sensor 8 in the x direction due to the rotation of the mirror 4, and 22 is an image of the input image 2 that has also been shifted by two pixels. 23, 24, 2
5 and 26 each indicate a light receiving section of the image sensor 8. Image 20 on light receiving sections 23, 24, 25, 26
Let the brightness of the images be X ₁ , X ₂ , X ₃ , and X ₄ , respectively. At this time, assuming that the pixel range of the weighted average is 3 pixels, the filtering output Y ₂ of the image at the position corresponding to X ₂ is obtained as follows: Y ₂ = ₃ 〓 ⁱ⁼¹ a _i X _i (2). Here, a _i is the weight for each pixel. In FIG. 3, first, the aforementioned variable transmittance filter 15 is adjusted to expose an image 20 to the image sensor 8 with a light amount proportional to the weighting coefficient _a1 . At this time,
The electric charges accumulated in the light receiving sections 23 _, 24, 25 _, and 26 are proportional to _a1X1 , _a1X2 , _a1X3 , and _{a1X4 , respectively} . Next, by rotating the mirror 4, the image 20 is shifted by one pixel pitch and moved to the position of the image 21,
Further, the transmittance of the variable transmittance filter 15 is changed to provide an image 21 with a light amount proportional to a ₂ and to accumulate charges. Since the charges are accumulated by adding them to the initial charges, the charges accumulated in the light receiving sections 23, 24, and 25 are respectively a ₁ X ₁ + a ₂ X ₂ , a ₁ X ₂ + a ₂ X ₃ , and a ₁ X ₃ +
Proportional to a ₂ X ₄ . Further, by rotating the mirror 4, the image 21 is shifted by one more pixel pitch and moved to the position of the image 22, and the image 22 is provided with a light amount proportional to a ₃ .The charges accumulated in the light receiving sections 23 and 24 are respectively a ₁ X ₁ + a ₂ X ₂ + a ₃ X ₃ , proportional to a ₁ X ₂ + a ₂ X ₃ + a ₃ X ₄ . That is, it can be seen that the electric charge of the light receiving section 23 is proportional to the filtering output _Y2 shown by equation (2).
Therefore, if this charge is read out, a filtering output corresponding to X ₂ will be obtained. Meanwhile, the filtering output Y ₃ corresponding to X ₃
is Y ₃ = ₃ 〓 ⁱ⁼¹ a _i X _i+1 (3), which is obtained from the amount of charge accumulated in the light receiving section 24. In this way, the entire image can be filtered by three exposures and two image shifts. In FIG. 1, a mirror 5 is rotated by a motor 7 and images an input image 2 on an image sensor 8.
Shift in the direction. Therefore, by performing the same operation as shown in FIG. 3, two-dimensional filtering is possible. The principle of such two-dimensional filtering will be explained with reference to FIG. FIG. 4 is a schematic plan view of the image sensor 8, and 30 indicates an image of an input image on the image sensor. 31, 32, 3
3, 34, 35, 36, 37, 38, and 39 are light receiving parts of the image sensor 8, respectively, and the brightness of each part of the corresponding image 30 is expressed as X ₁ , X ₂ ,
Let X ₃ , X ₄ , X ₅ , X ₆ , X ₇ , X ₈ , and X ₉ . here,
Two-dimensional filtering is performed by sequentially shifting the image 30 and capturing it with the image sensor 8. FIG. 5 is a schematic diagram showing the order of image shifting in FIG. 4. 40 is a movement curve of the image 30 on the image sensor 8. A single arrow between the circles represents movement by one pixel pitch. When the image 30 moves on the image sensor 8 along the movement curve 40, the light receiving section 31 receives images of X ₁ , X ₂ , X ₃ , X ₄ ,
Each portion of the image 30 is exposed in the order of X ₅ , X ₆ , X ₇ , X ₈ , and X ₉ . Therefore, by controlling the variable transmittance filter 15, the amount of light of the image 30 given to the image sensor 8 is proportional to a ₁ , a ₂ , a ₃ , a ₆ , a ₅ , a ₄ , a ₇ , a ₈ , and a ₉ in order. If so, the charge accumulated in the light receiving section 31 will be
₉ 〓 ⁱ⁼¹ a _i X Proportional to _i . This is the result of processing a weighted average within 3×3 pixels for _X5 . Since this kind of charge accumulation is performed on all the light receiving parts of the image sensor 8 at the same time, 9 exposures and 8 image shifts result in a 3×3 charge accumulation.
Filtering using a mask is performed over the entire image. The two-dimensional filtering described above is executed in the apparatus shown in FIG. 1 as follows. The necessary control signals are provided by controller 14. First, the image sensor driver 11 sweeps out the charge from the image sensor 8 in response to a signal from the controller 14, and exposure is started. At this time, the controller 14 controls the variable transmittance filter 15,
Transmits the amount of light proportional to a ₁ . After exposing the image sensor 8 to the transmitted light, the controller 14 sends a signal to the motor driver 9, which causes the motor 6 to move the mirror 4 so that the image of the input image 2 is displayed on the image sensor 8 by one pixel in the x direction. Rotate it so that it shifts by a pitch. At the same time, the controller 14
switches the transmittance of the variable transmittance filter 15,
A light amount proportional to a ₂ is applied to the image sensor 8.
Further, the mirror 4 shifts the image in the y direction by one more pixel pitch in accordance with the signal from the controller 14,
Through similar light amount control, the image sensor 8 is exposed with a light amount proportional to a ₃ . Next, the controller 14 controls the motor driver 10
The motor 7 rotates the mirror 5 in accordance with the drive signal from the motor driver 10 so that the image of the input image 2 is shifted by one pixel pitch in the y direction. After giving an image with a light intensity proportional to a ₆ at this position, the controller 14 again sends a signal to the motor driver 9, and the motor 6 causes the image of the input image 2 to
mirror 4 so that it shifts by one pixel pitch in the direction
Rotate. Here, the image sensor 8 has
An image is given with an amount of light proportional to a ₅ . Thereafter, when the image is similarly shifted in the x and y directions by rotating the mirrors 4 and 5 and exposure is repeated, the two-dimensional filtered image signal of the input image 2 is transmitted to the light receiving section of the image sensor 8. Accumulated. After the entire exposure is completed, the controller 14 gives a readout start signal to the image sensor driver 11,
An image signal 12 is read out from the image sensor 8. The image signal 12 is amplified by an amplifier 13 and sent to an appropriate processing circuit at the next stage. Other embodiments of the present invention will be described below. In the drawings, the same members as in FIG. 1 are designated by the same reference numerals, and detailed explanations will be omitted. FIG. 6 is a perspective view showing a second embodiment of the invention. 41 and 42 are parallel plates for shifting the input image 2 in the x direction and the y direction, respectively;
It is made of transparent material such as glass or plastic. 43 and 44 are parallel plates 41 and 44, respectively.
This is a motor for driving 42. The motor driver, image sensor driver, variable transmittance filter, and controller are the same as those shown in the first embodiment, so they are not shown. In this second embodiment, the image of the input image 2 is shifted by rotating the parallel plates 41 and 42. FIG. 7 is a front view showing the principle of image shift due to rotation of parallel plates. Here, 45 is an imaging light beam that comes out from one point of the input image 2 and forms an image on the image sensor 8, and 46 is an image point. Also, 47
shows a state in which the parallel plate 41 is rotated. 48 is an imaging light beam for the rotated parallel plate 47;
9 is an image point of the imaging light beam 48. On the image sensor 8 , an imaging light beam 45 from a certain point in the input image 2 passes through the parallel plate 41 and focuses on an image point 46 . On the other hand, with respect to the rotated parallel plate 47, the imaging light beam 48 is
Of course, the light passes through the same optical path as the imaging light beam 45 until it enters the beam 7, but after the light enters, it forms an image at an image point 49. Image point 49 is shifted in the x direction with respect to image point 46. Now, the parallel plate 41 has been rotated about an axis perpendicular to the plane of the paper, but if it is rotated about an axis parallel to the plane of the paper, the image point will shift in the y direction. In FIG. 6, a parallel plate 41 is rotated by a motor 43 and an image of an input image 2 is transferred to an image sensor 8.
Shift up in the x direction. On the other hand, the parallel plate 42 is rotated by a motor 44 to shift the image in the y direction. Filtering in the second embodiment is carried out in exactly the same manner as in the first embodiment by repeating the image shift and exposure process in which an amount of light proportional to the weighting coefficient a _i is applied to the image sensor 8. FIG. 8 is a perspective view showing a third embodiment of the present invention. In this embodiment as well, only the image shifting means is illustrated as in FIG. 6. 50 ₁ and 50 ₂ are glass plates, and 51 is a transparent elastic body sandwiched between the glass plates 50 ₁ and 50 ₂ . Examples of materials for the transparent elastic body 51 include silicone rubber 36 (trade name:
KE104Gel (manufactured by Shin-Etsu Chemical) is suitable, and adhesion to the glass plates 50 ₁ and 50 ₂ can be performed using a silane coupling agent. The glass plates 50 ₁ and 50 ₂ are parallel to the y-axis and form a certain angle with each other to form a prism together with the transparent elastic body 51. 52
₁ and 52 ₂ are glass plates, and 53 is the same transparent elastic body as 51. Glass plates 52 ₁ and 52 ₂ are x
It is parallel to the axis and forms a prism together with the transparent elastic body 53. The transparent elastic bodies 51 and 53 are glass plates 50 ₁ and 50
₂ or 52 ₁ , 52 ₂ , the prism can be made into a variable apex angle prism in order to change its shape accordingly. It is well known that when a prism is inserted into the optical path of an imaging optical system, the light beam will deflect and the image will shift according to its apex angle, and as the prism apex angle changes, the image will shift continuously. shift. The prism formed by the glass plates 50 ₁ and 50 ₂ and the transparent elastic body 51 changes its apex angle by moving the glass plate 50 ₂ by a drive system (not shown), and converts the image of the input image 2 into an image. x on sensor 8
Shift in the direction. On the other hand, glass plates 52 ₁ , 52 ₂
The prism formed by the transparent elastic body 53 changes its apex angle by moving the glass plate ₅₂₂ by a drive system (not shown), and changes the image of the input image 2 to y.
Shift in the direction. The filtering in the third embodiment is also the first,
Image shift and weighting coefficient a _i in exactly the same way as in the second embodiment
This is done by repeating the process of exposing the image sensor 8 to an amount of light proportional to the amount of light. FIG. 9 is a schematic cross-sectional view showing another example of the configuration of the variable apex angle prism. Reference numeral 60 indicates a plano-concave lens, and 61 indicates a plano-convex lens, and the curvature of the concave surface of the plano-concave lens 60 and the curvature of the convex surface of the plano-convex lens 61 differ only in sign but have the same absolute value. If the concave surface of the plano-concave lens 60 and the convex surface of the plano-convex lens 61 are aligned and the plano-convex lens 61 is slightly shifted, it will act as a prism with an apex angle of φ ₁ as shown in FIG. 9a, and if the plano-convex lens 61 is further shifted, it will become as shown in FIG. 9b acts as a prism with an apex angle of φ ₂ .
Even with such a configuration, a variable apex angle prism can be realized. Therefore, even if this prism is inserted into the imaging optical path shown in the third embodiment instead of the variable apex angle prism using a transparent elastic body, filtering can be performed in the same way. FIG. 10 is a schematic sectional view showing only the image shifting means of the fourth embodiment of the present invention. 70 ₁ and 70 ₂ are electro-optical materials such as BSO, LiNbO ₃ , KDP, and PLZT.
etc. are used. 71 ₁ , 71 ₂ , 71 ₃ , 71 ₄
is a transparent electrode, and ITO or the like is suitable. 73
is a birefringent material such as calcite, and 74 is also a birefringent material, but its thickness is half that of the birefringent material 73. 72 is incident light, and 75 and 76 are ordinary and extraordinary rays in the birefringent material 73 corresponding to the incident light 72, respectively. 77 and 78 are birefringent materials 74 corresponding to the ordinary ray 75, respectively.
79 and 80 are the ordinary ray and extraordinary ray in the birefringence material 74 corresponding to the extraordinary ray 76, respectively. 81, 82, 83,
Reference numerals 84 indicate outgoing light of the ordinary ray 77, the extraordinary ray 78, the ordinary ray 79, and the extraordinary ray 80, respectively. Transparent electrodes 71 ₁ are provided on both sides of the electro-optic material 70 ₁ ,
71 ₂ is provided, and by applying a voltage of about 1 KV to this, the electro-optic material 70 ₁ acts as a polarization rotation element. In other words, the incident light 72 is linearly polarized, and the direction of polarization is set by the electro-optic material 70.
The two main axes of ₁ are incident at an angle of 45° to each other. When the voltages applied to the transparent electrodes 71 ₁ and 71 ₂ are appropriately controlled, the light emitted from the electro-optic material 70 ₁ becomes linearly polarized light that is the same as the incident light 72 or linearly polarized in a polarization direction perpendicular thereto. The light emitted from the electro-optic material 70 ₁ becomes the ordinary ray 75 or the extraordinary ray 76 of the birefringent material 73 depending on its polarization direction, and in the case of the extraordinary ray 76, the birefringent material 7
When it is refracted at the surface of ray 6, it does not follow the usual Snell's law and travels in a direction different from that of ordinary ray 75.
Therefore, when the light is emitted from the birefringent material 73, the ordinary ray 75 and the extraordinary ray 76 are emitted from different positions. The electro-optic material 70 ₂ has transparent electrodes 71 ₃ , 71 ₄
is provided, and by applying a voltage, the incident linearly polarized light is converted into linearly polarized light with the polarization direction rotated by 90°. The polarization direction of the ordinary ray 75 is controlled by the electro-optic material 70 ₂ and becomes an ordinary ray 77 or an extraordinary ray 78 in the birefringent material 74 . Similar to the propagation in the birefringent material 73, the ordinary ray 77 and the extraordinary ray 78 travel in different directions in the birefringent material 74 and exit from different positions. However, since the thickness of the birefringent material 74 is half that of the birefringent material 73, the ordinary ray 77 upon exiting
The separation distance between the normal ray 78 and the extraordinary ray 78 is half that of the birefringent material 73. Similarly, the polarization direction of the extraordinary ray 76 is controlled by the electro-optic material 70 ₂ and the birefringent material 74
Inside, it becomes an ordinary ray 79 or an extraordinary ray 80.
The separation distance between the ordinary ray 79 and the extraordinary ray 80 upon emission is the same as the separation distance between the ordinary ray 77 and the extraordinary ray 78. The outgoing lights 81, 82, 83, and 84 are shifted by the same distance from each other, and the outgoing lights 81, 82, 83, and 84 are shifted by the same distance.
Which of 82, 83, and 84 to use is determined by the combination of voltages applied to the electro-optic materials 70 ₁ and 70 ₂ , for example, as shown in the table below.

〔Effect of the invention〕

As explained above, according to the image processing device of the present invention, arithmetic processing is performed on the image sensor, so there is no need for a computer or special hardware to process the output electrical signals, and the device configuration is simplified. , which makes it cheaper. Furthermore, since processing is performed in parallel for all pixels, the processing time remains unchanged even if the number of pixels increases, and even a large amount of image information can be processed at high speed.

[Brief explanation of the drawing]

FIG. 1 is a schematic diagram showing a first embodiment of an image processing device of the present invention, FIG. 2 is a plan view showing a part of an image, and FIG. 3 is a schematic diagram of an image sensor illustrating the principle of filtering according to the present invention. 4 is a plan view of an image sensor showing the principle of two-dimensional filtering, FIG. 5 is a schematic diagram showing the order of image shifting in FIG. 4, and FIG. 6 is a second embodiment of the present invention. 7 is a schematic cross-sectional view illustrating the principle of image shifting in the apparatus of FIG. 6; FIG. 8 is a schematic diagram showing an image shifting means of a third embodiment of the present invention; 9a and 9b are schematic sectional views showing other configuration examples of the variable apex angle prism in the apparatus shown in FIG. 8; FIG. 11 and 12 are plan views showing the positional relationship between pixels and light receiving sections in the device of the present invention, and FIGS. 13a and 13b are diagrams showing images before and after filtering processing, respectively. 2... Input image, 3... Lens, 4, 5... Mirror, 6, 7... Motor, 8... Image sensor, 9, 10... Motor driver, 11... Image sensor driver, 12... Electricity Signal, 13...
...Amplifier, 14...Controller, 15...Variable transmittance filter.

Claims (1)

[Claims] 1. An image processing device that converts an image into an electrical signal, comprising: a solid-state image sensor; a means for exposing the image to the solid-state image sensor multiple times while changing its relative position; and changing an amount of exposure light during the multiple exposures. and means for extracting charges induced by each exposure and accumulated in the solid-state image sensor as the electrical signals.