JP2001069521A - Signal processor, image pickup device, signal processing system, signal processing method and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device that can realize efficient signal processing and reduce the signal processing time by configuring an optimum delay line, depending on the purpose of signal processing. SOLUTION: A control means 213 groups n-sets of storage means 201-206 into (n-m) sets (n, m are integers) on the basis of signal processing by a signal processing means 105. Control selection of a storage means among n-sets of the storage means 201-206 for write-read and read an image signal with respect to the divided grouped storage means are imparted with a delay means 104 to function as act line for (n-m) delay lines.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing apparatus, an image pickup apparatus, a signal processing system, a signal processing method, and a signal processing method used for a delay line in digital image signal processing in a digital camera, for example. For a computer-readable storage medium.

[0002]

2. Description of the Related Art Conventionally, for example, in a digital camera, sensors having various color filter arrangements have been used.

FIGS. 17A to 17C show an example of a color filter array in the above sensor. FIG. 17 above
(A) is a 2 × 2 color filter array, and (b) is a 2 × 4 color filter array.
(C) shows a 2 × 8 color filter array.

[0004]

[0007] Incidentally, FIG.
When performing color processing on a signal output from a sensor having a color filter array as shown in each of (a) to (c), a color interpolation filter corresponding to the color filter array used in the sensor is used. There is a need to. That is,
When performing color processing on a signal output from the sensor having the 2 × 2 color filter array shown in FIG. 17A, it is necessary to use a three-tap color interpolation filter in the vertical direction. ), When performing color processing on a signal output from a sensor having a 2 × 4 color filter array, it is necessary to use a 5-tap color interpolation filter in the vertical direction.
When performing color processing on a signal output from a sensor having a 2 × 8 color filter array shown in FIG. 3C, it is necessary to use a 7-tap color interpolation filter in the vertical direction.

Accordingly, when signal processing (digital image signal processing including color processing) corresponding to the three types of color filter arrangements shown in FIGS. 17A to 17C is realized by hardware such as an IC, As shown in FIG. 18, before the signal processing circuit 910 for performing the digital image signal processing, 6H (horizontal synchronization) corresponding to the maximum number of taps in the vertical direction (here, 7 taps) of the color interpolation filter to be used is provided. Delay lines 921 to 926 are provided.

However, the delay line 9 as described above
In general, the RAMs 21 to 926 are realized by a RAM, and the capacity thereof is determined by the number of sample link bits when converting a signal output from a sensor from analog to digital (A / D conversion), Since it is proportional to the horizontal size at which the image signal processing is performed, if the digital camera is to be improved in image quality and definition, the two parameters of the number of sampling bits and the horizontal size are correspondingly increased. When a delay line is provided inside an IC for processing, the number of gates of the IC increases with an increase in the parameter. Therefore, in this case, there is a problem that the manufacturing cost of the IC increases.

On the other hand, instead of providing a delay line using a RAM inside the IC, a delay line using a DRAM or the like may be provided outside the IC. In this case, however, power consumption is higher than when the delay line is realized using a RAM. There was an increasing problem.

Therefore, in order to solve the above-mentioned problems, after the signal output from the sensor is A / D converted,
A method has been proposed in which a RAM such as a delay line provided inside an IC is reduced in scale by temporarily storing the data in a memory such as a DRAM, dividing the data into rectangular blocks, and performing signal processing. With this method, the manufacturing cost of the IC can be reduced.

However, in the above method, since the signal after A / D conversion is temporarily stored in a memory such as a DRAM, for example, when an ON display operation of an EVF (electronic viewfinder) is performed by a digital camera, the There is a problem that the time lag between the time when the image is photographed and the time when the photographed image obtained is displayed becomes large.

In order to solve the above-mentioned problem of the time lag, it is conceivable to perform digital image signal processing on a signal after A / D conversion without passing through a memory such as a DRAM and realize a 6H delay line. However, for this purpose, it is necessary to provide a large RAM.

Therefore, the present invention has been made to eliminate the above-mentioned drawbacks, and realizes efficient signal processing by configuring an optimal delay line according to the purpose of signal processing. The signal processing time was shortened,
It is an object of the present invention to provide a signal processing device, an imaging device, a signal processing system, a signal processing method, and a storage medium in which processing steps for executing the signal processing device are readable by a computer.

[0012]

For such a purpose,
A first invention is a signal processing apparatus comprising: delay means for delaying an input image signal by an arbitrary number of lines and outputting the signal; and signal processing means for performing signal processing on an output of the delay means. The delay means selects n (n: an integer) storage means for performing writing and reading of the input image signal, and selects the storage means to be read out of the input image signal from the n storage means. Means, and control means for controlling writing and reading of the input image signal to and from the n storage means and selection by the selection means, wherein the control means performs signal processing by the signal processing means. Based on the above n
Storage means are grouped into (nm) pieces (m: an integer), and writing and reading of the input image signal and reading of the input image signal are performed on the grouped storage means. By controlling the plurality of storage means and the selection means so that the selection of the storage means is performed, the delay means has a function as a (nm) delay line.

According to a second aspect of the present invention, in the first aspect,
The signal processing means performs filter processing of (n-m + 1) taps on the input image signal in a vertical direction.

According to a third aspect, in the first aspect,
The control means of the delay means performs control based on the input image signal so that writing is performed only in an arbitrary one of the n storage means.

A fourth invention comprises a delay means for delaying an image signal obtained by imaging by an arbitrary number of lines and outputting the signal, and a signal processing means for performing signal processing on the output of the delay means. In an imaging apparatus, the delay unit and the signal processing unit include the delay unit and the signal processing unit of the signal processing unit according to any one of claims 1 to 3.

A fifth aspect of the present invention is a signal processing system in which a plurality of devices are communicably connected, wherein at least one device of the plurality of devices is according to any one of claims 1 to 3. It has a function of a signal processing device or the imaging device according to claim 4.

According to a sixth aspect of the present invention, there is provided a signal processing method for delaying an image signal by an arbitrary number of lines by n (n: integer) storage means when performing an arbitrary signal processing on the image signal. A grouping step of grouping the n storage units based on the content of the signal processing;
For the group of storage means obtained in the grouping step, writing and reading of the image signal are controlled, and a storage means for reading out the image signal from the n storage means is selected. And a control step of controlling.

According to a seventh aspect, in the sixth aspect,
The control step includes a first count step of counting up each time the image signal is input and resetting the count value to a predetermined value, and a count up every time the count value of the first count step is reset. Then, the second count step of resetting the count value to a predetermined value, and the count values of the first count step and the second count step are decoded, and a plurality of state information set in advance are set. A state determining step of determining the corresponding state information from the above, and controlling the writing and reading of the image signal based on the state information determined in the state determining step and the count value in the first counting step. By controlling the selection of the storage means for reading out the image signal from the n storage means, (N-m) line: characterized in that it comprises a delay line control step of performing a delay line operation control (m an integer).

According to an eighth aspect, in the seventh aspect,
The delay control step includes, during a delay line operation of n lines (m = 0), delaying by a predetermined number equal to or less than L (L: integer) that can be stored in the storage means at the maximum, and (nm) L × n during line delay line operation
/ (N−m) or less by a predetermined number.

According to a ninth aspect, in the seventh aspect,
The control step is performed when the delay line operation is performed.
A series of operations for one data input includes a step of performing writing to only one of the n storage units.

According to a tenth aspect of the present invention, there is provided a signal processing apparatus according to any one of claims 1 to 3, or an image pickup apparatus according to claim 4, or a processing for implementing the function of the signal processing system according to claim 5. The program is a storage medium in which a computer readablely stores the program.

According to an eleventh aspect of the present invention, a computer readable storage medium stores the processing steps of the signal processing method according to any one of the sixth to ninth aspects.

More specifically, for example, n (n is an integer) storage means, and control of which one of the storage means is to be selected as a write / read operation to the n storage means and an output of the delay line. Control means for performing the operation. Further, the control means counts up each time data is input and is reset to "0" by a predetermined value. The control means counts up and resets each time the first count means is reset. There is provided second counting means which is reset to "0" by a value. The control means decodes the outputs of the first counting means and the second counting means to determine a state, and determines the state from the state and the output of the first counting means. Is generated, and a delay line operation of (nm) lines is performed by the signal. When the delay line operation of n lines (m = 0) is performed, L or less L units that can be stored in the storage unit at the maximum can be used. In the delay line operation of the (nm) lines, the delay is performed by a predetermined number of L × n / (nm) or less. With such a configuration, when the number of delay lines is small, an image signal having a larger horizontal size can be delayed.

[0024]

Embodiments of the present invention will be described below with reference to the drawings.

The present invention is applied to, for example, an imaging apparatus 100 as shown in FIG. The imaging apparatus 100 is an apparatus applied to a digital camera or the like, and as shown in FIG. 1, an imaging element (here, an imaging device that converts an image of a subject to be imaged into an electric signal (image signal) and outputs the signal. , "C
A CD sensor 101), an S / H and CDS circuit 102 for executing processing for clamping, amplifying, and sampling and holding the image signal from the CCD sensor 101;
A / D converter 103 for digitizing the image signal after processing in the / H & CDS circuit 102, and A / D converter 103
And a variable delay line 10 for delaying and outputting the image signal selected by the selector 113.
4 and YCr from the image signal from the variable delay line 104
A signal processing circuit 105 that performs signal processing such as generating a Cb signal (luminance signal and color difference signal); and a signal processing circuit 1
05, an image compression circuit 107 for converting the image signal processed by the enlargement / reduction circuit 106 into a signal of a predetermined size and aspect, an image compression circuit 107, a signal processing circuit 105, Scaling circuit 106,
And a DRAM 1108 used for temporarily storing the image signal for each processing on the image signal in the image compression circuit 107, and a DRAM controller 109 for controlling data writing and reading operations to and from the DRAM 1108.
And a display unit 111 such as an LCD or a monitor, and a DRA.
A display control circuit 110 that converts data read from the DRAM 108 by the M controller 109 into data in a signal format that can be displayed on the display unit 111; and an adder 112 that adds the data read from the DRAM 108 via the DRAM controller 109. And a selector 114 for selecting the output of the DRAM controller 109 or the output of the adder 112 as one input of the selector 113.

The most characteristic configuration of the image pickup apparatus 100 as described above is the configuration of the variable delay line 104, and the number of delay lines in the variable delay line 104 can be changed by setting.

The variable delay line 104 includes, for example, six RAMs (RAM (# 1) 201, R
AM (# 2) 202, RAM (# 3) 203, RAM
(# 4) 204, RAM (# 5) 205, RAM (#
6) 206), a selector 207 for selecting 1H delay data from the outputs of the six RAMs 201 to 206,
A selector 208 for selecting data with a 2H delay from the outputs of the six RAMs 201 to 206;
A selector 209 for selecting data with a 3H delay from the outputs of 1 to 296, and a selector 210 for selecting data with a 4H delay from the outputs of the six RAMs 201 to 206.
A selector 211 for selecting data with a 5H delay from the outputs of the six RAMs 201 to 206;
A selector 212 for selecting 6H-delayed data from the outputs of 201 to 206, a delay line for generating an access address for the six RAMs 201 to 206, a read / write control signal, and a selection control signal for the six selectors 207 to 212 And a controller 213.

The variable delay line 104 as described above has two
H, 4H, and 6H can operate with three delay line numbers.

In this case, the number of sampling bits in the A / D converter 103 is 10 bits, so that the RAM (# 1) 201, the RAM (# 2) 202,
RAM (# 3) 203, RAM (# 4) 204, RAM
<# 5) 205 and RAM (# 6) 206 are all 640 words with a 10-bit width, that is, 10 (bit) × 640 (word) = 6400 (bi)
t) memory capacity.

The selection control signals for the six selectors 207 to 212 (the selection control signals generated by the delay controller 213) are 6-bit signals.
It operates in accordance with a 6-bit control signal from 13. For example, when bit 5 of the selection control signal is “H”, RA
The output of M (# 6) 206 is selected, and when bit 4 is "H", the output of RAM (# 5) 205 is selected, and bit 3 is selected.
Is "H", the output of the RAM (# 4) 204 is selected, when the bit 2 is "H", the output of the RAM (# 3) 203 is selected, and when the bit 1 is "H", the output of the RAM (# 4) is selected.
2) The output of 202 is selected, the output of RAM (# 1) 201 is selected when bit 0 is "H", and "0" is output when all bits are 0.

In the selection control signals to the six selectors 207 to 212, two or more bits do not become "H" at the same time.

The delay line controller 213 has a configuration as shown in FIG.
An access address and a Read / Write control signal for the two RAMs 201 (# 1) to 206 (# 6) and a selection control signal for the six selectors 207 to 212 are generated.

First, the variable delay line 1 shown in FIG.
The data input to 04 (the image signal output from the selector 113: see FIG. 1 described above) is provided with a flag bit for determining whether the data of the data cycle is valid or invalid. .

Therefore, in the delay line controller 213, first, the data input detection circuit 301 detects the flag bit added to the input data, and the H counter 302 (binary counter) detects the flag bit.
Each time a flag bit is detected, the counter is incremented by one.

In the register 304, the value of (horizontal size -1) of the input data to the variable delay line 104 is set in advance by a CPU (not shown) which controls the operation of the entire apparatus.

The comparator 303 (comparator) compares the count value of the H counter 302 with a value preset in the register 304. As a result of the comparison, if they match, an "H" level signal is output. , And outputs an “L” level signal if they do not match. The output of the comparator 303 is supplied to the V counter 305 and also to the H counter 302.

The H counter 303 includes a comparator 303
Is reset to "0" when the output becomes "H".

On the other hand, when the output of the comparator 303 becomes "H", the V counter 305 (binary counter)
Increment the counter by one. That is, the V counter 305 counts up every time the H counter 303 is reset.

The register 307 includes the register 3 described above.
As in the case of the apparatus 04, a predetermined value is set in advance by a CPU (not shown) that controls the operation of the entire apparatus. For example, "5" is set in the 6H delay line operation, "3" is set in the 4H delay line operation, and "1" is set in the 2H delay line operation.

The comparator 306 (comparator) compares the count value of the V counter 305 with a value preset in the register 307, and as a result of the comparison, when they match, an “H” level signal , And outputs an “L” level signal if they do not match. The output of the comparator 306 is supplied to a V counter 305.

The V counter 305 includes a comparator 303
Is "H" and the output of the comparator 306 is "H", it is reset to "0".

The output of the V counter 305 is supplied to a V count decoder 308, where it is decoded. Specifically, for example, as shown in FIG. 4, the V count decoder 308 converts the output value into a decoded value (binary) according to the output of the V counter 305. This decoded value is
The control signal decoder 314 for the 6H delay line is supplied to the control signal decoder 315 for the 4H delay line, and the control signal decoder 316 for the 2H delay line.

Control signal decoder 31 for 6H delay line
4, 4H delay line control signal decoders 315 and 2
The H delay line control signal decoder 316 has an output of the V count decoder 308 and the H counter 30 described above.
2 and comparators 311-309 as follows:
Is also provided.

The comparator 309 includes an H counter 302
Is compared with the set value of the register 312,
As a result of the comparison, if the count value of the H counter 302 is smaller than the set value of the register 312, an “H” level signal is output; otherwise, an “L” level signal is output.

The comparator 310 includes an H counter 302
Are compared with the set value of the register 312 and the set value of the register 313. As a result of the comparison, when the count value of the H counter 302 is equal to or larger than the set value of the register 312 and smaller than the set value of the register 313, Outputs an "H" level signal, otherwise outputs an "L" level signal.

The comparator 311 has an H counter 302
Is compared with the set value of the register 313,
As a result of the comparison, if the count value of the H counter 302 is equal to or more than the set value of the register 313, an "H" level signal is output; otherwise, an "L" level signal is output.

The comparators 309 to 311 as described above
The setting values of the registers 312 and 313 to be compared in the RAM (##) used in the variable delay line 104
1) The value depends on the capacity of the 201 to RAM (# 6) 206, and is used to determine which RAM is to be used based on the current count value of the H counter 302.

Specifically, here, the RAM (# 1) 2
01 to RAM (# 6) 206 are set to 10 (bit) × 640 (word) as described above.
At the time of the delay line operation, as shown in FIG.
1) Since the 201 to the RAM (# 6) 206 are used by dividing them into six delay lines, the registers 312 and 313 each have a set value of the register 312: 640 (the number of words in the RAM) and a set value of the register 313: 640 (Number of words in RAM) is set.

In the 4H delay line operation, as shown in FIG. 6, the RAM (# 1) 201-RAM (# 6) 2
06 is divided into four delay lines and used, so the registers 312 and 313 each contain the setting value of the register 312: 320 (half the number of words in the RAM) and the setting value of the register 313: 640 (the number of words in the RAM). Set.

In FIG. 6, "RAM (# 1)
"Lower than 201" indicates an address space smaller than the middle of the address space of the RAM (# 1) 201, that is, a memory area of an address space from 0 to 319.
“Higher than RAM (# 1) 201” means RAM (# 1)
Above the middle of the address space 201, i.e. 320-
6 shows a memory area of an address space up to 639. Other R
For AM, “lower” and “higher” have the same meaning.

In the 2H delay line operation, as shown in FIG. 7, the RAM (# 1) 201 to the RAM (# 6) 2
06 is divided into two delay lines and used, so the registers 312 and 313 each have a setting value of the register 312: 640 (the number of words in the RAM) and a setting value of the register 313: 1280 (twice the number of words in the RAM). Set.

The comparators 311 to 30 as described above
The control signal decoder 314 for the 6H delay line, the control signal decoder 315 for the 4H delay line, and the control signal decoder 316 for the 2H delay line to which the output of the H counter 302 and the output of the V count decoder 308 are supplied together with the output of A selector control signal (selection control signal), a Read / Write control signal, and an access address signal are generated from the supplied signals.

More specifically, for example, as shown in FIG. 8, the 6H delay line control signal decoder 314 outputs the outputs of the V count decoder 308 to the State6H-1 to St6
ate6H-2, and as shown in FIG. 9, the selector control signal, Read / W
The write control signal and the access address are decoded.

The 4H line control signal decoder 315 is
As shown in FIG. 10, the output of the V count decoder 308 and the outputs of the comparators 309 to 311 are used to calculate St.
The states of the ate4H-1 to State4H-12 are determined, and as shown in FIG. 11, the selector control signal, the read / write control signal, and the access address are decoded according to the determination result.

The 2H line control signal decoder 316
As shown in FIG. 12, St output is obtained from the output of the V count decoder 308 and the outputs of the comparators 309 to 311.
ate2H-1 to State2H-6,
As shown in FIG. 13, the selector control signal, the Read / Write control signal, and the access address are decoded according to the determination result.

The 6H delay line control signal decoder 314 and the 4H delay line control signal decoder 31 as described above.
The selector control signal, the Read / Write control signal, and the access address signal generated by the 5 and 2H delay line control signal decoder 316
To 319.

The selector 317 includes a 6H delay line control signal decoder 314, a 4H delay line control signal decoder 315, and a 2H delay line control signal decoder 3.
The selector control signal generated at 16 is supplied. The selector 317 controls the 6H delay line control signal decoder 3 according to the mode set in the mode register 320.
One of the fourteen selector control signals, the selector control signal of the 4H delay line control signal decoder 315 and the selector control signal of the 2H delay line control signal decoder 316, is selected and output.

The selector control signal output from the selector 317 is a signal having a width of 36 bits.
bit35 to bit30 are control signals of the selector 207,
bit29 to bit24 are control signals of the selector 208,
bit23 to bit18 are control signals of the selector 209,
bit17 to bit12 are control signals of the selector 210,
bit11 to bit6 are control signals of the selector 211, b
It5 to bit0 are composed of control signals for the selector 212.

The selector 318 includes a 6H delay line control signal decoder 314, a 4H delay line control signal decoder 315, and a 2H delay line control signal decoder 3
The read / write control signal generated at 16 is supplied. Selector 318 selects mode register 320
The read / write control signal of the 6H delay line control signal decoder 314, 4
Read / W of control signal decoder 315 for H delay line
A write control signal and one of the read / write control signals of the 2H delay line control signal decoder 316 are selected and output.

The output of Read /
The Write control signal is a signal that indicates a “Write after Read” operation at “H” level and a “Read” operation at “L” level, which will be described in detail later. Read / Write timing signal generator 321
Are RAM (# 1) 201 to RAM (# 6) 2 according to the Read / Write control signal from the selector 318.
A Read / Write signal for 06 is generated.

The selector 319 includes a 6H delay line control signal decoder 314, a 4H delay line control signal decoder 315, and a 2H delay line control signal decoder 3
The selector address generated at 16 is supplied. The selector 319 controls the 6H delay line control signal decoder 3 according to the mode set in the mode register 320.
One of 14 selector addresses, the selector address of the 4H delay line control signal decoder 315, and the selector address of the 2H delay line control signal decoder 316 are selected and output.

As described above, the selector control signal, the read / write control signal, and the access address are selected by the selector 317, the selector 318, and the selector 319, respectively.
4, the number of delay lines is changed.

The selector control signal, the read / write control signal, and the access address obtained by the variable delay line 104 are stored in the selectors 207 to 212 and the RA.
M (# 1) 201 to RAM (# 6) 206 (see FIG. 1). That is, the selector 317
Control signal (selection control signal) selected by
Are supplied to the selectors 207 to 212, respectively, and are based on the Read / Write control signal selected by the selector 318.
The output of the write timing signal generator 321) and the access address selected by the selector 319 are:
The signals are supplied to the RAM (# 1) 201 to the RAM (# 6) 206, respectively.

Here, the RAM (# 1) 201-RAM
(# 6) The Read / Write operation for 206 includes a “Write after Read” operation for writing new data after reading the delayed data, and a “Read” operation for reading only the delayed data.

As described above, which of these operations is executed depends on whether the RAM (# 1) 201 to the RAM (# 6)
It is determined by a Read / Write control signal (output of the selector 318) which is a basis of the Read / Write signal supplied to the control signal 206. That is, when the Read / Write control signal selected by the selector 318 is an “H” level signal, “Write after
This indicates execution of a “Read” operation. When the signal is at “L” level, it indicates execution of a “Read” operation.
Read / Wri based on ad / Write control signal
The te signal is between RAM (# 1) 201 and RAM (# 6) 2
06 as a signal for controlling the writing and reading operations of the RAM (#
1) It is supplied to 201 to RAM (# 6) 206.

The Read / Write signal is a 6-bit signal, and bit 0 is the read signal of the RAM (# 1) 201.
d / Write signal, bit 1 is RAM (# 2) 202
Read / Write signal, bit 2 is RAM (#
3) Read / Write signal of 203, bit 3 is R
Read / Write signal of AM (# 4) 204, bit 4 is Read / Write signal of RAM (# 5) 205, bit 5 is Read / Wr of RAM (# 6) 206
ITE signal.

RAM (# 1) 201-RAM (# 6) 2
06, for example, SCRAM (Single port Clocke)
d RAM), the “Write a
The “after Read” operation and the “Read” operation
As shown in FIG.

In FIG. 14, "CLK" is the SCR
A clock to AM, an “address” is an access address to the SCRAM (an output of the selector 319),
"Data input" is data to be written to the SCRAM.
The “Write / Read signal” is the same as the above-mentioned Wr
a signal based on the item / Read control signal,
“H” indicates reading, and “L” indicates writing. Also,
“Data output” is data output from the SCRAM. In addition, from time t1 to time t3, “Writ
e after Read "operation or" Read "
This is one cycle of operation.

Therefore, "Write after Rea"
In the d ”operation, first, at time t1, the SCRAM
Is an access address (An), a data input (Dn), a Write /
Read signal is latched. And SCRAM is
When it is determined that the Write / Read signal is at the “H” level, the delay data (Qn) of the access address (An) is read and output.

Next, at time t2, the SCRAM
At the rise of the clock (CLK), the access address (An), the data input (Dn), and the Read / Wri
latch the te signal. And SCRAM is Wr
When it is determined that the item / Read signal is at “L” level, the data input (D) is input to the access address (An).
Write n). Therefore, SCRA at this time t2
The output data of M becomes the valid output of the variable delay line 104.

On the other hand, in the “Read” operation, Write
Since the / Read signal is always at the “L” level, the SCR
AM is the delay data (Q) of the access address (An).
This is an operation that only reads and outputs n).

As described above, the RAMs (# 1) 201 to R (# 1) 201 to R are determined by the Write / Read signal and the access address obtained by the delay line controller 213.
The writing and reading operations of AM (# 6) 206 are controlled.

The delay line controller 21 as described above
The number of delay lines set for the variable delay line 104 including 3 differs depending on the purpose of signal processing. For example,
When the imaging apparatus 100 is applied to a digital camera, the purpose of performing signal processing by the signal processing circuit 105 is as follows: EVF: Determines a composition to be photographed.・ Record: Record the image of the subject obtained by shooting. -Recorded image review: Check the image obtained by shooting. Etc.

The signal processing path and the number of delay lines for the purpose of the above-described signal processing will be described below. Note that, in the following description, the CCD sensor 101
00 pixels, 1200 lines in the vertical direction.
It is assumed that it is composed of one million pixels. In the variable delay line 104, 10 (bit) × 640 (worn)
RAM (# 1) 201-RA having a memory capacity of d)
It is assumed that six RAMs of M (# 6) 206 are used.

(1) At the time of EVF operation At the time of EVF operation, it is necessary to perform signal processing in real time. Further, in order to display the image signal after the signal processing in the signal processing circuit 105 on the display unit 111 such as an LCD or a monitor, it is attempted to read out all 1600 pixels in the horizontal direction and 1200 pixels in the vertical direction from the CCD sensor 101. Then, it is difficult to obtain a sufficient frame rate. Further, since the image signal after the signal processing in the signal processing circuit 105 is reduced for the VRAM by the enlargement / reduction circuit 106, it is not necessary to read all the pixels.

Therefore, during the EVF operation, first, when an image signal is read from the CCD sensor 101,
Addition and thinning of pixel signals are performed.

For example, the color filter array of the CCD sensor 101 is changed to a 2 × color filter as shown in FIG.
When the color filter array of 8 is used, the image signal output from the CCD sensor 101 is read by adding and thinning out as shown in FIG. I do.

That is, during the EVF operation, From the relationship shown by the following formula, the CCD sensor 101
The image signal output from the
An image signal having an array of × 2 is obtained.

The image signal output from the CCD sensor 101 as described above is digitized by the A / D converter 103. At this time, the input to the selector 113 is selected as the output of the A / D converter 103 by the CPU (not shown) that controls the operation of the entire apparatus described above. As a result, the output of the A / D converter 103 is input to the variable delay line 104 via the selector 113.

In the variable delay line 104, the image signal from the selector 113 is delayed by the above-described configuration. In the signal processing circuit 105 to which the delayed image signal in the variable delay line 104 is supplied, At the time of the EVF operation, image signals in a 2 × 2 arrangement are handled, so that signal processing such as color processing using a 3-tap color interpolation filter in the vertical direction is performed.

For this reason, the variable delay line 104 is set in advance by the CPU so as to be a 2H delay line (2H delay line operation). 2H
In the delay line operation, the variable delay line 104 includes six RAMs (# 1) 201 to (# 6) 206 as described above.
Is divided into two delay lines, so that it is possible to handle a maximum of 640 (word) × 3 (pieces) = 1920 pixel horizontal lines.
Signal processing can be performed without going through the 08.

The image signal delayed by the variable delay line 104 is subjected to signal processing by a signal processing circuit 105, and is output from the signal processing circuit 105 as a YCrCb signal.

The enlargement / reduction circuit 106 includes the signal processing circuit 10
Display unit 11 for the YCrCb signal output from
VRAM data is generated by performing an enlargement or reduction process so as to have a size and an aspect suitable for 1.

VRAM obtained by scaling circuit 106
The data is transferred to the DRA via the DRAM controller 107.
Written to M108. The VRAM data written in the DRAM 108 is read out by the display control circuit 110 via the DRAM controller 109, encoded into a video signal, and supplied to the display unit 111.

(2) At the time of recording operation At the time of recording operation, signal processing for recording an image signal obtained by photographing on a certain recording medium is performed.

Therefore, at the time of the recording operation, first, the image signal (digital image data) output from the A / D converter 103 is temporarily converted to a DR signal via the DRAM controller 109.
Write to AM108.

At this time, for example, when the color filter array of the CCD sensor 101 is a 2 × 8 color filter array as shown in FIG.
The image signal written in 08 has the same arrangement as the color filter arrangement of the CCD sensor 101.

Therefore, the signal processing circuit 105 has 2 ×
That is, image signals having an arrangement of 8 are handled, and signal processing such as color processing using a 7-tap color interpolation filter is performed in the vertical direction.

The signal processing circuit 105 performs signal processing on the image signal arbitrarily divided into blocks as the signal processing.

Therefore, the variable delay line 104 is set in advance by the CPU so as to be a 6H delay line (6H delay line operation).

Further, the selector 11 is provided by the CPU.
The input to 3 is selected as the output of DRAM controller 109. Thereby, the DRAM controller 109
Output from the DRAM 108 via the variable delay line 10
4 is input. That is, an image signal is read from the DRAM 108 by the DRAM controller 109 in the form of divided blocks, and this image signal is supplied to the variable delay line 104.

The image signal delayed by the 6H delay line operation in the variable delay line 104 is supplied to the signal processing circuit 105, and is output as a YCrCb signal from the signal processing circuit 105.

The scaling circuit 106 includes the signal processing circuit 10
Enlargement or reduction processing is performed on the YCrCb signal output from 5 at a ratio suitable for recording on a recording medium (not shown).

The compression circuit 112 compresses the image signal processed by the enlargement / reduction circuit 106 and outputs the compressed image signal to the DRAM 10 via the DRAM controller 109.
Write to 8.

The image signal written in the DRAM 108 is recorded on the recording medium by the CPU.

(3) At the time of the recorded image review operation In the recorded image review operation, for example, an image signal (photographed image) once written in the DRAM 108 is displayed on the display unit 111 so that the photographer can confirm the photographed image. Performs signal processing for display.

In this recorded image review operation, first, A
The image signal (digital image data) output from the / D converter 103 is once written into the DRAM 108 via the DRAM controller 109.

The EVEN line and the CCD line of the image signal written in the DRAM 108 are read simultaneously by the DRAM controller 109. The read signal is supplied to the variable delay line 104 via the adder 112, the selector 113, and the selector 114.
At this time, the signal selection operation by the selector 113 and the selector 114 is performed by the CP that controls the operation control of the entire device described above.
U (not shown). As a result, the adder 112 applies the EV to the variable delay line 104.
A result obtained by adding the EN line and the CCD line or a result obtained by halving the added result is supplied.

The image signal supplied to the variable delay line 104 has, for example, a 2 × 4 arrangement as shown in FIG. 16, and therefore, the image signal delayed by the variable delay line 104 is supplied. In the signal processing circuit 105, signal processing such as color processing using a 5-tap color interpolation filter in the vertical direction is performed.

The signal processing circuit 105 performs the above-described signal processing in the same manner as the above-described (2) signal processing in the recording operation.
The signal processing is performed on the image signal divided into blocks.

Here, there are 4H delay and 6H delay as operation modes of the variable delay line 104 that can support signal processing in the signal processing circuit 105, that is, signal processing of 5 taps in the vertical direction.

As described above, in the 4H delay line operation, the six RAMs (# 1) 201 to (# 6) 206
Since it is divided into two delay lines and used, a maximum of 320 (word) × 3 = 960 pixels can be handled, so that signal processing can be performed on a divided image signal obtained by dividing an image into two blocks in the horizontal direction.

On the other hand, in the 6H delay line operation, six R
Since the AM (# 1) 201 to (# 6) 206 are used by dividing them into six delay lines, only 640 pixels can be handled at the maximum, so that a divided image signal obtained by dividing an image into three blocks in the horizontal direction is used. Needs to be performed.

Therefore, here, in order to perform signal processing in a shorter time, the variable delay line 104 is set beforehand by the CPU so as to perform the 4H delay line operation. The output of the variable delay line 104 operating according to this setting is subjected to signal processing in a signal processing circuit 105 to become a YCrCb signal,
Supplied to

The enlargement / reduction circuit 106 includes the signal processing circuit 10
5 is subjected to enlargement or reduction processing so as to have a size and an aspect suitable for display on the display unit 111 to generate VRAM data.

VRA generated by scaling circuit 106
M data is transferred to the DR controller 107 via the DRAM controller 107.
AM 108 is written.

The VRAM data written in the DRAM 108 is read by the display control circuit 110 via the DRAM controller 109, encoded into a video signal, and supplied to the display unit 111.

As described above, in this embodiment, the six RAMs (# 1) 201 to (# 6) 206 and their R
The write and read operations of the AM (# 1) 201 to (# 6) 206 are controlled, and the output of the delay line
(# 1) Delay line controller 213 for controlling which RAM output from 201 to (# 6) 206 is selected
The variable delay line 104 including the signal processing circuit 10
In response to the signal processing at 5, it operates on any of the 2H delay line, 4H delay line, and 6H delay line.

That is, when the signal processing in the signal processing circuit 105 in accordance with the purpose (EVF, recording, recorded image review, etc.) is filter processing with seven taps in the vertical direction, six RAMs (# 1) 201 to (# 6) 206 is 6
By dividing them into two groups, they operate as six delay lines (the number of RAMs is n = 6 and m = 0, and (6-0 = 6) delay lines). In some cases, six RAM
By dividing (# 1) 201 to (# 6) 206 into four groups including their address spaces, they operate as four delay lines (the number of RAMs n = 6, m =
In the case of 2 ((6-2 = 4) delay lines), if the filter processing is three taps in the vertical direction, the six RAMs (# 1) 201 to (# 6) 206 are divided into two groups. As a result, it operates as a two-line delay line (the number of RAMs n = 6, m = 4, (6
-4 = 2) line delay line).

Therefore, for example, when signal processing is performed without passing through the DRAM 108 during the EVF operation in the CCD sensor 101 of 1600 pixels in the horizontal direction and 1200 lines in the vertical direction, in the conventional configuration, 10 (bit) × 1600 (word) On the other hand, a RAM having a memory capacity of 10 (b) × 1600/3 (word) ≒ 10 (b) is required in the configuration of the present embodiment.
A RAM having a memory capacity of (it) × 534 (word) is sufficient.

In addition, VR
Since the AM data can be created, the time lag from when the image is captured to when the image is displayed on the display unit 111 is reduced as compared with the related art.

When performing signal processing for a recorded image review, the adder 112 performs line addition.
The number of lines for performing signal processing can be halved. This reduces signal processing time by half.

In the 4H delay line operation, the horizontal size that can be handled is 1.5 times as large as that in the 6H delay line operation, so that the number of image divisions also decreases.

In the prior art, as shown in FIG. 18, when one data is input, the operation of writing the output of the other RAM is performed. Every time is input, the writing operation is performed in all the RAMs. On the other hand, in the configuration according to the present embodiment, when one data is input, the writing operation is performed on only one RAM, so that the power consumption of the RAM is reduced as compared with the related art.

It is to be noted that an object of the present invention is to supply a storage medium storing program codes of software for realizing the functions of the host and the terminal of the above-described embodiment to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. It is needless to say that the present invention can also be achieved by a CPU or an MPU) reading and executing a program code stored in a storage medium. In this case, the program code itself read from the storage medium implements the functions of the present embodiment, and the storage medium storing the program code constitutes the present invention. ROM, as a storage medium for supplying the program code,
Floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape,
A non-volatile memory card or the like can be used. By executing the program code read out by the computer, not only the functions of the present embodiment are realized, but also the OS and the like running on the computer perform actual processing based on the instructions of the program code. It goes without saying that a part or all of the above is performed, and the processing realizes the function of the present embodiment. Further, after the program code read from the storage medium is written to a memory provided in an extension function board inserted into the computer or a function extension unit connected to the computer, the function extension is performed based on the instruction of the program code. It goes without saying that a CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the present embodiment.

[0116]

As described above, according to the present invention, it is possible to provide a delay line optimized for the purpose of performing signal processing. Further, this makes it possible to efficiently use the memory as a limited delay line that can be mounted inside the IC, and reduce the number of gates of the IC. As a result, the development cost of the IC is reduced.

[Brief description of the drawings]

FIG. 1 is a diagram for describing a configuration of an imaging device to which the present invention has been applied.

FIG. 2 is a block diagram illustrating a configuration of a variable delay line of the imaging device.

FIG. 3 is a block diagram showing a configuration of a delay line controller of the variable delay line.

FIG. 4 is a diagram for explaining a decoding method of a V count decoder in the delay line controller.

FIG. 5 is a diagram for explaining a method of using a RAM at the time of a 6H delay line operation in the variable delay line.

FIG. 6 is a diagram for explaining a method of using a RAM at the time of a 4H delay line operation in the variable delay line.

FIG. 7 is a diagram for explaining a method of using a RAM in a 2H delay line operation in the variable delay line.

FIG. 8 is a diagram for explaining a decoding method of a state during the 6H delay line operation.

FIG. 9 is a diagram for explaining a method of decoding each control signal during the 6H delay line operation.

FIG. 10 is a diagram for explaining a decoding method of a state during the 4H delay line operation.

FIG. 11 is a diagram for explaining a method of decoding each control signal during the 4H delay line operation.

FIG. 12 is a diagram for explaining a method of decoding a state during the 2H delay line operation.

FIG. 13 is a diagram for explaining a method of decoding each control signal during the 2H delay line operation.

FIG. 14 is a diagram for explaining access timing of the RAM.

FIG. 15 is a diagram illustrating a CC during an EVF operation in the imaging apparatus.
It is a figure for explaining the data arrangement at the time of reading out data for EVF by a D sensor.

FIG. 16 is a diagram for explaining a data array after addition by an adder during a recorded image review operation in the imaging apparatus.

FIG. 17 is a diagram illustrating an example of a color filter array of a CCD sensor.

FIG. 18 is a block diagram showing a configuration of a conventional delay line.

[Explanation of symbols]

 Reference Signs List 100 imaging device 101 CCD 102 S / H & CDS circuit 103 A / D converter 104 variable delay line 105 signal processing circuit 106 enlargement / reduction circuit 107 image compression circuit 108 SRAM 109 DRAM controller 110 display control circuit 111 display unit 112 adders 113, 114 Selector 201-206 RAM 207-212 Selector 213 Delay line controller

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) KE20 KE24 KM02 KM13

Claims (11)

[Claims] 1. A signal processing apparatus comprising: delay means for delaying an input image signal by an arbitrary number of lines and outputting the signal; and signal processing means for performing signal processing on an output of the delay means. The means includes n (n: an integer) storage means for performing writing and reading of the input image signal, and selecting means for selecting the storage means from which the input image signal is read out from the n storage means. And control means for controlling writing and reading of the input image signal to and from the n storage means and selection by the selection means, wherein the control means is based on signal processing by the signal processing means. The above-mentioned n storage means are grouped into (nm) pieces (m: integer), and writing and reading of the input image signal and reading and writing of the input image signal are performed on the grouped storage means. Memory to be read A signal processing apparatus characterized in that the plurality of storage means and the selection means are controlled so as to select the means, so that the delay means has a function as a (nm) delay line. . 2. The signal processing apparatus according to claim 1, wherein said signal processing means performs a filtering process of (n−m + 1) taps on the input image signal in a vertical direction. 3. The control means of the delay means performs control based on the input image signal so that writing is performed only in an arbitrary one of the n storage means. The signal processing device according to claim 1. 4. An image pickup apparatus comprising: delay means for delaying an image signal obtained by imaging by an arbitrary number of lines and outputting the signal; and signal processing means for performing signal processing on an output of the delay means. An imaging apparatus, wherein the delay unit and the signal processing unit include the delay unit and the signal processing unit of the signal processing unit according to claim 1. 5. A signal processing system in which a plurality of devices are communicably connected, wherein at least one device among the plurality of devices is the signal processing device according to claim 1, A signal processing system having the function of the imaging device according to claim 4. 6. When performing arbitrary signal processing on an image signal,
A signal processing method for delaying the image signal by an arbitrary number of lines by n (n: an integer) storage units, wherein the n storage units are grouped based on the content of the signal processing. Controlling the writing and reading of the image signal with respect to the group of storage means obtained by the grouping step; and reading the image signal from the n storage means. And a control step of controlling the selection of the storage means. 7. The control step includes counting up each time the image signal is input, resetting the count value to a predetermined value, and resetting the count value in the first count step. A second count step of counting up each time the count value is reset and resetting the count value to a predetermined value; and decoding each count value of the first count step and the second count step. Determining the corresponding state information from among the state information of the above, and writing and reading the image signal based on the state information determined by the state determining step and the count value by the first counting step. Of the storage means for reading out the image signal from the n storage means. By controlling, (n-m) line (m: integer) signal processing method according to claim 6, characterized in that it comprises a delay line control step of performing a delay line operation control. 8. The delay control step includes, during an n-line (m = 0) delay line operation, delaying by a predetermined number of L (L: integer) or less that can be stored in the storage means at the maximum. At the time of the (nm) line delay line operation, L × n /
8. The signal processing method according to claim 7, further comprising the step of: delaying by a predetermined number of (nm) or less. 9. The control step includes a step of performing writing to only one of the n storage units in a series of operations for one data input when performing the delay line operation. The signal processing method according to claim 7, comprising: 10. A computer-readable storage medium storing a processing program for implementing the functions of the signal processing device according to claim 1, the imaging device according to claim 4, or the signal processing system according to claim 5. A storage medium characterized by being stored in a readable manner. 11. A storage medium, wherein the processing steps of the signal processing method according to claim 6 are stored in a computer-readable manner.