JPH0350984A - Picture recorder - Google Patents

Abstract

PURPOSE:To simplify a signal processing circuit and to facilitate circuit integration by providing a matrix arithmetic means converting an input signal into a prescribed chrominance or luminance signal. CONSTITUTION:An output of an image pickup element 1 enters a matrix circuit B having a white balance adjustment function via a matrix circuit A converting the output into r, g, b signals and then enters an NTSC standard TV signal processing encoder 4 via a gamma correction circuit 2 and a color difference matrix circuit 3. On the other hand, as to the luminance signal processing system, the output of the element 1 is given to a matrix circuit D converting it into a luminance signal and a high frequency component is extracted via a gamma correction circuit 5 and an HPF 6, synthesized with a low frequency luminance signal YL from the circuit 3 at an adder 7 and converted into a luminance signal Y. That is, the information of matrix being color signal matrix G=AXB is enough to obtain a color signal required from the element 1 and the presence of the matrix D is enough to obtain the luminance signal.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of application in RA industry] The present invention relates to an image recording device using a solid-state image sensor combined with a color separation filter.

[Prior art] A signal from a solid-state image sensor is converted into a digital signal, and the signal is converted into a digital signal.
2. Description of the Related Art Devices that perform digital signal processing to record and reproduce video are attracting attention. A typical example of this is an all-solid-state camera that converts video information into digital values and records them in a semiconductor memory or the like. In the above system, the signals output from the image sensor need to undergo various signal processing depending on the arrangement of the color separation filter attached to the image sensor (hereinafter abbreviated as sensor). In advance, on the camera side, the signal is converted into a standard signal format (for example, a luminance signal and a color difference signal) regardless of the arrangement of color separation filters, and is recorded in a semiconductor memory or the like.

The sensor signal then has a -degree standard form (e.g. R
, G, B signals) and then recorded.

[Problem that the invention seeks to solve] However,
The conventional example described above has the disadvantage that the amount of information that should originally be recorded is increased and recorded in the storage memory.For example, the output of a sensor with a mosaic filter as shown in FIG. When capturing the luminance signal into memory, the memory capacity for the luminance signal is effectively the same as the number of pixels on the sensor, since the sampling frequency cannot be lowered.Furthermore, the color difference signal, which was originally obtained line-sequentially, is required simultaneously. Because the sensor needs to be converted into an image and stored, it requires several times the capacity of the original sensor. Of course, it is possible to lower the sampling frequency for color difference signals. However, in that case, a digital filter is required to lower the sampling frequency, and since there is a limit to lowering the sampling frequency, it is inevitable that the information will increase significantly compared to the original sensor information. This also applies to a sensor having a stripe filter as shown in FIG. In order to store the information in memory without increasing the amount of information, it is possible to record the sensor information as it is, but in that case, there may be major problems with compatibility due to differences in signal processing due to the arrangement of various color filters. There are drawbacks that leave problems.

[Means for Solving the Problems] The present invention is aimed at solving the problems of the conventional example described above. The present invention is characterized in that it has a selection means for selecting block data of an arbitrary size from the above, and a matrix calculation means for converting the selected signal into a predetermined color signal or luminance signal.

As a specific example, the playback device is provided with an n×m matrix signal processing means that converts the sensor signals into RGB and luminance signals, respectively, and when the recording device stores the sensor signals in the memory, the sensor signals are stored as directly as possible. Along with letting
By recording the conversion matrix constant of the matrix signal processing, the grid array information of the sensor, the number of vertical and horizontal movement steps of the conversion matrix on the sensor, etc. in the memory, the same This makes it possible to obtain a signal output in the form of

[Function] According to the present invention, even if sensor signals are almost directly written into memory, compatibility is not impaired, so
Memory can be saved.

[Example] First, the principle of matrix signal processing, which is the key point of the present invention, will be explained. The types of color separation filters used in single-chip and dual-chip cameras are limited to complementary colors and pure colors. , and then converts it into the required signal format.

For example, in a complementary color mosaic filter as shown in FIG. 1, such conversion is performed in a signal processing circuit as shown in FIG. In other words, regarding color signal processing, the output of the image sensor 1 is passed through a matrix circuit A that converts it into RGB,
Matrix circuit with white balance adjustment function,
NTSC via γ correction circuit 2 and color difference matrix circuit 3
(PAL, SECAM, HD, etc.) into the standard television signal processing encoder 4. On the other hand, in the luminance signal processing system, the output of the image sensor 1 passes through a matrix circuit that converts it into a luminance signal, passes through a γ correction circuit 5 and a digital filter 6, and high-frequency components are extracted.
The adder 7 synthesizes the low-band luminance signal from Y
is converted to In other words, in order to obtain the necessary color signal from the perspective of the image sensor, all that is needed is information on the color signal matrix G=AXB, and in order to obtain the luminance signal,
It is sufficient that matrix D exists. In actual signal processing, further vertical and horizontal sampling steps are required. In other words, since calculations are performed by fitting an l×m matrix onto the pixels of the image sensor 1, information on which pixel position the matrix is to be moved to next is required. The calculation process will be explained using Figure 1. First, the signals of each pixel of the image sensor 1 are arranged in xxy on the memory RAM so that they can be accessed randomly (or the data necessary for matrix calculations can be appropriately extracted using a delay line, etc.). ). At this time, using the window matrix Gc shown in FIG. 2, this Gc is x
The coordinates (1, 1) on xy are first set, and r, g, b, and Y are calculated by the matrix operation shown in FIG.

Next, this window matrix is moved according to a sampling step prepared in advance. For example, in the example shown in Figure 1, the window matrix is (1
,1) becomes (3,1) after two cycles.
), and this movement is set in advance,
This can be done by sampling step=2.

Also, if you set the (xy) size of the sensor in advance, you can stop the window matrix when it exceeds that coordinate, and after the cycle corresponding to IH, the X coordinate will be changed again. , return to the initial position and move the window matrix by a preset vertical sampling step for y, thereby performing signal processing for the next line.

The same applies to the luminance signal, and the signal processing matrix D prepared in advance is used for the window matrix.
By calculating , a luminance signal is obtained, and by moving this window matrix in sampling steps, signal processing similar to that for color can be performed. In this way, by preparing a window matrix and a signal processing matrix, mosaic filters and stripe filters, signal processing of pure colors and complementary colors, which were conventionally completely different signal processing forms, can be easily performed using the same hardware form. Processing can now be done simply by changing the matrix. In Figure 3,
FIG. 2 shows a signal processing block diagram of the present invention including a window matrix. The same reference numerals as in FIG. 2 indicate the same elements. Note that the RAM is a fixed or removable random access memory. Figure 4 shows an example of pure color stripe filter arrangement and the matrix at this time.
Adaptation to the case of FIG. 4 in the hardware configuration shown in the figure will be described.

In the case of a pure color filter, the matrix A portion in FIG. 4 is a 3×3 unit matrix as shown in FIG. 4, and the luminance signal is also represented by a 3×1 matrix. This can also be applied to the case where line signals are mixed to obtain Y (No. 2) with the aim of increasing the sensitivity of the sensor in movies etc. In this case, the window matrix Gc is , in the Y direction, and as a result, the array elements of D can also be lengthened, and Y, or Y2 can be selected and output for each field.

Of course, in this case, if initial coordinates for each field of the window mask are newly prepared as information, this matrix can be reduced. In the case of the Y system, if this window matrix is made large, it is possible to output it as a vertical enhancer signal, and it goes without saying that this operation can be included in the luminance matrix.

FIG. 6 shows an example of an all-solid-state camera that utilizes the principle of the present invention described above, and its operation will be explained below. 7
1 is an optical system lens group, 72 is an aperture, 73 is an infrared cut filter, 74 is an optical low-pass filter 75
is an on-chip color separation filter, and the image passing through the systems 71 to 75 is focused on a solid-state image sensor 76, where it is converted into an electric charge. The converted charge is transferred to the 97 system signal generator (hereinafter referred to as 5SG).
) is read out in synchronization with the signal from ) and enters the variable gain amplifier 77. The variable gain amplifier 77 is for sensitivity correction when the aperture 72 cannot be fully adjusted, and this amplifier also performs KNEE correction processing at the same time. 78 is an optical sensor for fixing the black level of the sensor.
This is a black clamp circuit, and the signal whose DC level is fixed here is converted into a digital value by an A/D converter 79, and sequentially stored in the memory 80 at the address indicated by the memory controller 98. Note that the memory 80 may be removable or fixed. 96
is an average level detection circuit for controlling exposure using the image sensor output, and by reading this value, the aperture 72
and controls the variable gain amplifier 77 to adjust the exposure level. In the Naoki embodiment, the electronic shutter operation is performed by controlling the drive timing of the image sensor 76. Further, in this embodiment, the format when writing to the memory is as shown in FIG. 7. That is, as shown in FIG. 7, in the header file, the number of currently recorded images, their titles, and, if something other than images is recorded, their attributes are digitized and recorded. Further, an ID file section is provided at the beginning of each image No. 48 recording area. ID
The file portion records matrix information of the signal processing matrices 81 and 82, and information necessary for the window mask, such as the grid shape of the sensor and the sampling step.

Subsequently, image data is recorded. When playing,
First, the system controller reads the ID through the memory controller 98, and transfers the matrix constant and digital filter constant number to the matrix arithmetic processing circuit 81 and 82 and the digital filter 95.
Furthermore, the wind mask information and sampling step are
8's memory controller, and the playback status is now set. During playback, the 98 memory controller
A window mask is formed, and information corresponding to the window position is transferred to the color signal processing matrix circuit 81 and the luminance signal processing matrix circuit 82 in accordance with the sampling step, and here, based on the matrix information transferred from the ID file in advance, , matrix calculation signal processing is performed to form a luminance signal and a color difference signal. The formed luminance signal then enters a digital filter 95, where only high-frequency components are extracted and sent to an adder 94.
The correction circuit 1 calculated by the color signal processing matrix 81
04, combined with the low-range luminance signal YL obtained via the matrix circuit 105, then subjected to blanking processing in the blanking circuit 85 according to the signal from the system signal controller 100 during playback, and then the adder 93 A synchronizing signal is added at , and converted to an analog signal by a D/A converter 88 .
The sampling carrier is dropped and output as a luminance signal via an amplifier 92.

On the other hand, the color difference signal output from the matrix circuit 105 is first
83 and 84 blanking circuits add blanking pulses, then an NTSC encoder 86 converts it into a chroma signal, then a D/A converter 87 converts it into an analog signal, passes through a bandpass filter 89 and an amplifier 91. It is output as a color signal C. Note that a digital filter may be inserted after the color signal processing matrix circuit 81.

FIG. 8 is a diagram showing an example of a 3×4 71-risk among the n×m window matrices in the present invention,
This window matrix allows necessary data to be obtained from the random access memory, and is associated with d+-d+z of the signal processing matrix. The specific signal format is a 1-bit signal as shown in Figure 8, which is used to select the relevant data, and also to set the start coordinates as No. 13 that controls this window mask. information such as sampling steps and other information is required.

The signals obtained in this way are as shown in Figure 9.
A matrix operation as shown in FIG. 10 is performed manually in a ×12 signal processing circuit, and output as ", g, bfa".

At this time, the values 81 to a24 are sensor-specific values,
Furthermore, this constant is a value that includes white balance information, signal processing method, color filter characteristics, etc.

Note that although we have talked about the window matrix as n×m, this is not limited to a rectangle; the shape of the window matrix may be any shape of window with one element. . The same can be said of the luminance signal processing matrix.

FIG. 1) is a diagram showing an example of a matrix circuit for luminance signal processing, and this example shows an example of a 2×12 matrix. )'+ and y2 indicate the results after signal processing of the first field and the second field, respectively. These results are switched and output for each field by a multiplexer MPX. Also, the constant blNb at this time
Specifically, 24 is a constant indicating information such as Y-system white balance, brightness level difference correction information, and vertical enhancer.

This luminance signal matrix can also be composed of k×j (k and j are natural numbers respectively), and the present invention can be composed of 2 as in this embodiment.
It is not limited to ×12. Furthermore, if 1(xj and mxn are the same), the latch in front of the signal processing circuit can be shared, and if the signal processing circuit operates at a sufficiently high speed, they can be made the same and operated in time division.

In the embodiment, the data is directly stored in the memory RAM, but it goes without saying that the data may be compressed and stored and expanded during playback.

[Effects of the Invention] According to the present invention, signal processing circuits that conventionally differed due to the arrangement of color separation filters of an image sensor can be configured with a common circuit, and when integrated into an IC, the mass production effect is improved. You can expect it. Furthermore, if a solid-state camera is manufactured using the present invention, compatibility will not be compromised even if the sensor signal is directly captured into the memory, so it is possible to record the sensor signal directly into the memory as a recording format. This can improve memory usage efficiency.

[Brief explanation of drawings]

Fig. 1 is a diagram showing an example of arrangement of complementary color mosaic filters and an example of matrix type, Fig. 2 is a diagram for explaining the present invention in detail, Fig. 3 is a diagram showing an example of the configuration of the present invention, Fig. 4 5 is a diagram showing an example of the arrangement of pure color stripe filters and an example of a matrix type. FIG. 5 is a diagram showing another example of a brightness matrix type for pure color stripe filters. FIG. 6 is a diagram showing a specific example of the image recording apparatus of the present invention. 7 is a diagram showing a configuration of a memory space; FIG. 8 is a diagram showing another example of a window matrix; FIG. 9 is a diagram showing an example of a color signal processing matrix circuit; This figure is a diagram showing an example of the formula of the color signal matrix in FIG. 9, and FIG. 1) is a diagram showing an example of the luminance signal processing matrix circuit. l? °° →

Claims (3)

[Claims] (1) A color signal from a solid-state image sensor is stored in a memory without color conversion, and has a selection means for selecting block data of an arbitrary size from the memory, and a predetermined block data is selected from the selected signal. An image recording device characterized by having a matrix calculation means for converting into a color signal or a luminance signal. (2) In the image recording apparatus, the matrix calculation means includes an n×m luminance signal processing matrix calculation processing means and a k×j color signal processing matrix calculation processing means. 1) Image recording device. (3) In addition to the image signal, the memory contains information regarding the size of block data used for matrix calculation, matrix calculation constants, data regarding the shape of the pixels of the solid-state image sensor, data regarding the pixel arrangement, and data from the arrangement. Digital filter constants for correcting the resulting phase difference, table of nonlinear processing means used during playback,
The image recording device according to claim (1), further comprising means for recording at least one of a constant of a color difference matrix, a luminance, and a constant of a digital filter for band limiting the color difference signal. .