JPH0515116B2 - - Google Patents

Description

[Detailed Description of the Invention] (a) Industrial Application Field The present invention is applicable to images from an image sensor or memory of a video camera or an electronic still camera used to express an image by horizontal scanning, such as in a television. The present invention relates to a two-dimensional image signal processing device that improves image quality by effectively performing, for example, band limiting and edge enhancement processing by using a signal scanning direction conversion method.

(b) Technical field In recent years, there has been rapid progress in image processing, and improvements are being made to enable signal processing to obtain higher image quality, especially for two-dimensional images, using equipment that is as simple as possible and capable of performing various types of processing. For example, in video cameras and still cameras, color signals are obtained from imaging signals obtained using a single tube or plate, but when obtaining the color signal, various processes are required, such as processing for band limitation and edge enhancement. By doing this, the image is improved. Processing of two-dimensional images is difficult to configure, and when vertical false color signals are removed by the above-mentioned band-limiting processing for signals captured using an interlaced method, horizontal scanning lines are read out in the horizontal direction, and then one or two scanning lines are read out. The shifted signal must be extracted and processed. Even if the image was processed as described above, the image obtained was not perfect and some problems remained in the image.

Two-dimensional image processing will be described below as a process for removing false color signals in the vertical direction among various types of image processing.

Conventionally, various methods are known for color separation, and the color separation method used for single-tube cameras includes:
There are three electrode methods, phase separation methods, step energy demodulation methods, frequency separation methods, etc.

The three-electrode method separates colors by dividing the transparent signal electrode of the vidicon into three parts corresponding to the vertically arranged RGB stripe filters, and the three primary color signals are independent from the three separated signal electrodes. It has many advantages, such as easier signal processing, better color reproducibility, less noticeable color shading, and a wider dynamic range.However, the transparent signal electrode is divided into three parts. Therefore, it is difficult to manufacture electrodes.

In the phase separation method, a transparent signal electrode is divided into two parts corresponding to the vertically arranged RGB stripe filters, and an electron beam is scanned while applying an offset pulse to the two divided transparent signal electrodes.
A mixed signal of an RGB point-sequential video signal and a reference phase signal that serves as a reference for color separation is obtained, and by performing signal processing on this mixed signal, the video signal and the reference phase signal are separated, and the reference phase signal is separated from the reference phase signal. This method uses phase signals to detect color difference signals. Although it has advantages such as good white balance, less noticeable color shading, and wide dynamic range, it is still difficult to manufacture because it requires dividing the transparent signal electrode.

The step energy demodulation method uses stripe filters to perform color separation, and although signal processing is simple and electrode manufacturing is not difficult, it requires up to the second harmonic of the repetition frequency of the stripe filter for color separation. Therefore, there is a problem that the required bandwidth is larger than other methods.

The frequency separation method uses intersecting stripe filters as shown in Figure 1 to perform color separation by correlating the video signals obtained by two consecutive horizontal scans and performing addition and subtraction. The red and blue signals are modulated on the determined carrier wave and separated from the Y signal. This method has advantages such as good light utilization, high sensitivity, and easy manufacturing because it does not require dividing the transparent signal electrode, but it has problems such as color shading and dynamic range.

By the way, among the color separation methods of the single-tube camera mentioned above, the frequency separation method uses vertical correlation to create R, G, and B signals for each pixel, so it is difficult to see where the hue or brightness changes rapidly. False color signals may occur and the hue and brightness may change.

The problem of false color signal generation during color separation using vertical correlation also occurs when color separation is performed in a color camera using a single plate. Color separation in a single-chip color camera is usually done using a stripe filter in which R, G, B or Y, M, C, etc. are arranged alternately in the vertical direction, or a stripe filter in which R, G, B is arranged periodically in both the vertical and horizontal directions. A mosaic filter (for example, a Bayer array) that changes in color is used, and no matter which filter is used, the signal of each pixel is held until the next signal of the same color appears. However, when a stripe filter is used, there is no problem, but when a mosaic filter is used, false color signals occur in areas where the brightness or hue changes in the vertical direction for the same reason as in the case of a single tube.

This point will be explained in detail using FIGS. 2 and 3.

FIG. 2 shows a case where the brightness changes rapidly in the horizontal direction (from black to white), and the R, G, B and IQ signals, which are color signals, on the Nth scan are as shown in the same figure. In this case, the signal of a certain pixel is held until the next pixel of the same color, and for the signal of a color not present in that line, the signal of the previous line ((N-1)th scanning line) is used as is. The signal S _H that appears when the brightness changes in the I/Q signals is called the false color signal.
Current solid-state image sensors have approximately 350 to 400 horizontal picture elements and are read out using a 7.2MHz clock, so the frequency of this false color signal S _H is 3.85MHz. However,
Since the I signal is limited to 1.5 MHz and the Q signal is limited to 0.5 MHz, the false color signal S _H is removed. The same is true for single frequency separation image pickup tubes in which the currently widely used color filter has a color modulation frequency of 3.58 MHz. Furthermore, since the frequency of the false color signal is high in an image pickup plate with a large number of picture elements or an image pickup tube with a high color modulation frequency, the false color signal can be removed without any problem.

The problem of false color signals also occurs when brightness or hue changes in the vertical direction. FIG. 3 shows a case where the luminance changes rapidly in the vertical direction (from black to white), and the B, R, G, I and Q signals seen in the direction of the arrows are as shown in the figure. In this case as well, the I/Q signals include the false color signal _SV . Since the frequency of this false color signal S _V is a horizontal scanning frequency of 15.75 KHz, it cannot be removed by the band limit set for I/Q signals, and in the reproduced image, the white part at the boundary between black and white becomes magenta. It will be colored.

Therefore, instead of using the signal from one scan ago as it is, by using the average value of the signal from one scan before and the signal from the next scan, the result shown in Figure 3 is...
A method of creating a signal as indicated by I. has been proposed. According to this method, the level of the false color signal S′ _V is reduced to 1/2, but the width is doubled, so at the boundary from black to white in the reproduced image, the color becomes faint but blurred. The problem is that it doesn't meet the boundaries. Conventionally, this problem has been solved by compressing or expanding the color signal level according to changes in the vertical luminance signal level, but with this method, the luminance level does not change, only the hue changes. False color signals generated at the edges cannot be removed, and a complicated device is required to completely remove the false color signals in the vertical direction. In this way, when processing a two-dimensional image consisting of a large number of horizontal scanning lines, when the horizontal scanning lines are read out horizontally and processed, the processing in the vertical direction requires a complicated apparatus.

For example, a processing device for vertical false color signal removal and vertical edge enhancement becomes complicated. The same applies to other various image processing.

(C) Object and Structure of the Invention The present invention has been made in view of the problems in two-dimensional image processing as described above, and for example, eliminates vertical false color signals that occur when performing color separation using vertical correlation. It effectively performs processing such as vertical contour enhancement and vertical contour enhancement with a relatively simple configuration.
An object of the present invention is to obtain a two-dimensional image with improved image quality, and to perform various image processing in the vertical direction with a simple configuration in which signals from a memory or an image sensor are read out and processed in the vertical direction. was completed. Specifically, the present invention converts two-dimensional image information stored in an image sensor or memory into a vertical scanning signal, performs vertical response correction processing, and then converts it back into a horizontal scanning signal. achieved.

Now, the basic structure of the present invention is schematically shown in FIG. 4, where 1 is a two-dimensional image sensor or a memory that stores image signals scanned horizontally and read out from the image sensor. , which stores image signals. Reference numeral 2 denotes a signal processing section which processes image signals sent in the vertical direction and sends them out in the vertical direction. 3 is a two-dimensional memory, in which written signals can be read out in different scanning directions (for example, vertical signals are written and read out in the horizontal direction). The signals ○→ and ↓ shown beside the signal lines in the figure mean a state in which the signal is sent out in the horizontal direction and a state in which the signal is sent out in the vertical direction, respectively.

(d) Examples The present invention will be explained below based on the drawings.

FIG. 5 shows an example of the basic circuit configuration of signal processing when the present invention is applied to an electronic still camera, taking processing for removing vertical false color signals as an example. A shown surrounded by a broken line in the figure is an electronic still camera, B is a playback device, and the image pickup device and the image processing section are separate bodies. In the electronic still camera A, 4 is an optical lens that forms an optical image of the subject, and 5 is an analog image by photoelectrically converting the optical image (two-dimensional image information) formed on the light receiving surface by the optical lens 4. Generally known image pickup devices that output signals include image pickup tubes such as Sachicon, Vidicon, Plumbicon, Cosbicon, and Newcos Bicon, and solid-state image pickup devices such as CCD, MOS, and CPD. The above-mentioned intersecting stripe filter 5a is attached to the light receiving surface of the image sensor 5 so that a color image can be obtained. The image sensor 5 outputs an analog image signal based on the drive signal from the driver 6, and this image signal is input to the A/D
The converter 7 converts the signal into a digital signal at high speed. The image signal digitally converted by the A/D converter 7 is stored in the memory 10. In this embodiment, the memory 10 is removable from the electronic still camera A, and may be any memory that can stably store digital signals, especially N-MOS RAM,
Semiconductor memories such as C-MOS RAM and CCD memory are preferred, and in addition, magnetic valve elements,
A magnetic disk, magnetic tape, etc. may also be used.
The writing and reading of image signals into and from the memory 10 is controlled by the driver 8. Reading of analog image signals from the image sensor 5 by the driver 6 and writing of digital image signals into the memory 10 by the driver 8 are normally performed in the horizontal direction.

In playback machine B, 10 is electronic still camera A
9 has a memory in which two-dimensional image information is recorded, and 9 reads out the digital image information stored in the memory 10 and separates the colors into Y, I, and Q.
a processing circuit including a matrix that generates the three signals; 11;
A is a memory that sequentially stores I signals created by the processing circuit 9 in the horizontal direction, 11b is a memory that sequentially stores Q signals created by the processing circuit 9 in the horizontal direction, and 12a is a memory that sequentially stores the I signals created by the processing circuit 9 in the vertical direction. A filter 12b places a band limit on the I signals sequentially read out from the memory 11b, a filter 13a stores the I signal passed through the filter 12a in the vertical direction. memory 13b is filter 1
This is a memory that stores the Q signal that has passed through 2b in the vertical direction. That is, in FIG.
The part where the signals of a and 11b are processed by filters 12a and 12b and the read direction is converted by memories 13a and 13b corresponds to the basic configuration of the present invention shown in FIG. 4, and the filter is a part of the signal processing section. This is one of our implementations. In the above embodiment,
It is also possible to integrate the camera A and the playback device B in FIG. 5 and directly process the data read out in the vertical direction from the image sensor without going through the memory, as shown by the dashed line l. This has the effect of reducing memory, enabling high-speed processing, and is also effective for processing moving images.

As can be seen from the above circuit configuration, exactly the same signal processing is performed on the I signal and Q signal, so
The removal of false color signals according to the present invention will be explained using the I signal as an example. As shown in FIG. 6, the digital secondary image information stored in the memory 11a is read out in the vertical direction as shown. An analog filter 120 can be used as the filter 12a, and in this case, a digital analog (D/
A) Converter 121 and analog/digital (A/
D) converter 122 is required. Memory 11a
The digital image information read out in the vertical direction is converted into an analog image signal by the D/A converter 121, and band-limited by the analog filter 120. Bandwidth limitation by the analog filter 120 is determined by the frequency of the clock for reading out image information from the memory 11a, the resolution of the desired image quality, etc., and is preferably limited to about 1/4 of the readout clock frequency. Therefore, for example
When reading with a 7.2MHz clock, false color signals of 1.8MHz or higher will be cut out. The analog image signal whose band has been limited and false color signals have been removed is digitized by the A/D converter 122 and stored vertically in the memory 13a. According to this circuit configuration, if the readout speed of the memory 11a is slowed down, low-speed D/A and A/D converters can be used, resulting in low cost, and the processing speed is at a level that does not pose any practical problem for still image processing. It can be done. As explained above, according to the present invention, vertical false color signals generated when performing color separation using vertical correlation in a single-tube or single-chip color camera can be removed by providing vertical band limitation. As a result, sharp edges can be obtained without blurring or blurring of boundaries at edge portions where brightness or hue changes in the vertical direction. The present invention can be applied not only to still images taken with a still camera, but also to moving images taken with a video camera.

In addition to the analog filter described above, the filter 12a for removing false color signals caused by vertical correlation may be a digital filter, or a microcomputer or the like may be used to perform filter processing by software. Although filter processing using software has the disadvantage of slow speed, it has the advantage of being able to easily respond to changes in various processing conditions. Another advantage is that it can perform not only band limiting but also processing such as vertical edge enhancement. That is, in the present invention, by adopting the configuration shown in FIG. 4, response correction in the vertical direction of two-dimensional image information (two-dimensional image signal) can be very easily performed. It is possible to remove vertical false color signals by band-limiting the color signals (I, Q signals) through a low-pass filter, or to perform processing to enhance edges in the vertical direction, or to improve the memory after the processing section. It is also possible to perform interpolation processing on the image signal by setting the capacity to be larger, for example, twice the capacity of the image memory. Furthermore, as an image sensor, it is possible to perform correction processing to match characteristics caused by differences between interlaced and non-interlaced systems to visual characteristics, and to match signal characteristics in the horizontal scanning direction, such as band limit amount. It is also possible to perform processing to obtain an image with a high quality suitable for the human eye.

In FIG. 5, if the image information stored in the memory 13a is read out in the horizontal direction, an I signal from which the vertical false color signal has been removed can be obtained.

Although the explanation is omitted for the Q signal, the same band limiting process as for the above-mentioned I signal is performed.

In addition, in the NTSC system currently adopted as the transmission system for television signals in Japan, I,
Bandwidths of 1.5 MHz and 0.5 MHz are provided for the Q signal, respectively, but this means that the horizontal resolution is only 120 TV lines and 40 TV lines, respectively. This is sufficient because this is the only resolution of the human eye for I and Q signals. However, the I and Q signals also have a resolution of 525 lines in the vertical direction, which is of excessive quality compared to the horizontal resolution. Therefore, in the present invention, even if the band is limited in the vertical direction as described above, there is no problem with the image quality.

Now, as a device configuration for performing contour enhancement in the vertical direction in the above processing, a schematic configuration as shown in FIG. 7 is possible. In the figure, 10 is an image sensor or a memory that stores a two-dimensional image from the image sensor, and the contents of the signal processing section 9 vary depending on the type of signal from the memory or image sensor 10, but the signal from the image sensor may be used as is. For example, it is a color separation matrix, and if it is an NTSC signal, it is a demodulation circuit. The necessary condition is that the memory or image sensor 10 is combined with the signal processing section 9, and the signal processing section 9 outputs Y, I, and Q signals of horizontal scanning.
I/Q signals can be output and used as they are. memory 1
1c converts only the Y signal into vertical scanning, and the signal processing unit 12c converts [X _o ← (1+2h) X _o -h (X _o-1 +
X _o+1 )]. Here, h is a coefficient that determines the strength of edge enhancement. The obtained signal _Xo is converted into horizontal scanning in the memory 13c and output as a Y signal.

In the embodiment shown in FIG. 5, when a solid-state image sensor is used, two-dimensional image information is read out in the horizontal direction and stored in the memory, and then, after color separation processing, the I/Q signals are once stored in the memory in the horizontal direction, and then the two-dimensional image information is read out in the vertical direction. I read it out and performed band limit processing in the filter section, but
In addition, the image sensor or the image signal is directly read out vertically from the memory stored horizontally for each horizontal scanning line, and is stored vertically after being subjected to band-limiting processing using a filter. Alternatively, the data may be read out and played back.
FIG. 8 shows another example of the implementation configuration. In the figure, reference numeral 5 denotes an image sensor, and a signal processing unit 2 reads signals from the image sensor 5 in the vertical direction and performs color separation processing and other processes for improving image quality as described above.
a, and the vertical direction signal (Y, I/Q
(divided into signals) into memories 3a, 3b, 3c
After writing, it is read out horizontally.
The signal processing unit 2a is a circuit that processes the vertical scanning signal obtained from the image sensor 5. For example, in the case of vertical false color signal removal, it does nothing to Y and limits the band only to I and Q. . For contour enhancement, I.
Do nothing to Q and do X _o ←X _o (1+2h)-h(X _o-1
+X _o+1 ). A combination of these is also possible. In Fig. 8, Y is obtained from a general horizontal scanning video camera circuit without going through this processing circuit, and only I and Q are passed through this processing circuit to remove false color signals, and conversely, only Y is passed through this circuit to remove false color signals. , I.Q.
The configuration is such that vertical contour enhancement is performed by those that do not pass through. The basic configuration of the signal processing device according to the present invention will be described repeatedly, but as shown in FIG. In the previous stage, the signal can be read out vertically, processed, stored in a memory, read out horizontally, and output to recording means, display means, etc.

The above embodiment is an example of an electronic still camera, but
The method of the present invention can basically be applied to moving images. When applying it to videos, the processing time must be shortened, so it is necessary to use memory with high-speed access time and increase the hardware part.

(E) Effects of the Invention As described above, the signal processing device of the present invention, particularly the signal processing device for two-dimensional images, performs various processing on image signals read by conventional horizontal scanning in order to improve image quality. In the field of image processing, image signals are read in the vertical direction and various methods are applied to these vertical signals. By performing this processing, it has become possible to provide a very useful configuration of a signal processing device that is very effective and allows the user to easily change processing conditions and settings such as changes in specifications.

In particular, in a system that records images using an electronic still camera, when the recording medium is attached to and removed from the camera and image processing is performed on the playback device, it is possible to use a single playback device to record images with different characteristics from various cameras by setting the signal processing section. By switching manually or automatically, it is now possible to obtain an image that is suitable for each characteristic of the camera on the playback device side.

[Brief explanation of the drawing]

Fig. 1 is an example of a crossed stripe filter used in the frequency separation method, Fig. 2 is an explanatory diagram explaining the generation of horizontal false color signals, Fig. 3 is an explanatory diagram explaining the generation of vertical false color signals, and Fig. 4 is an explanatory diagram explaining the generation of vertical false color signals. The figure shows the basic configuration of the signal processing device of the present invention, and Figure 5 is a schematic diagram of the camera in the case where the present invention is applied to a playback device in an electronic still camera system and band limitation is performed in the vertical direction. FIG. 6 is a block diagram showing an example of the main part of the signal processing according to the present invention, and FIG. 7 is a block diagram showing another embodiment of the signal processing apparatus according to the present invention for vertical edge enhancement. This is an example of the configuration. FIG. 8 is a diagram showing a schematic configuration of still another embodiment. 1... Image sensor or memory storing image signals, 2... Signal processing unit, 3, 10, 11a, 11
b, 11c, 13a, 13b, 13c...memory, 4...optical lens, 5...imaging element, 9...
Processing circuit, 12a, 12b...Filter, 120
...Analog filter.

Claims (1)

[Claims] 1 Separation means that separates two-dimensional image information stored in an image sensor or memory into a luminance signal and a color signal, and a separation means that sequentially separates the color signals separated by the separation means in an order corresponding to the main scanning direction of the two-dimensional image. a first storage means for storing; a signal processing means for performing response correction processing on the color signals sequentially read out from the first storage means in an order corresponding to the sub-scanning direction of the two-dimensional image; and the signal processing a second storage means for sequentially storing the color signals subjected to response correction processing by the means in an order corresponding to the sub-scanning direction of the two-dimensional image;
A signal processing device for a two-dimensional image, comprising: reading means for reading out the color signals stored in the second storage means in an order corresponding to the main scanning direction of the two-dimensional image.