JP4310177B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method applied as, for example, a digital camera or the like, and is particularly applied to an applied technique when a high-speed image processing or a multifunctional camera system is configured.

  Conventionally, as an imaging apparatus applied as a digital camera or the like, for example, there is a digital still camera configured as shown in FIG. In the illustrated example, the overall control CPU 116 detects a change in state of a camera operation switch (not shown) (configured by the camera main SW and release SW) by the photographer himself and supplies power to the other circuit blocks. And make initial settings.

  The subject image within the imaging screen range is imaged on the image sensor (CCD in this case) 100 through the main imaging optical system 118, and an output signal from the image sensor 100 is sent to each of the images via the CDS / AGC / AD circuit 103. Each pixel is sequentially converted into a predetermined digital signal.

  Here, the image pickup means (image pickup element) 100 is based on a signal from a timing generator (TG / SSG) 102 that determines the overall drive timing, and a driver circuit 101 for horizontal drive and vertical drive for each pixel. The image signal output is generated by predetermined driving with the output of.

  The output signal of the CDS / AGC / AD circuit 103 is input to the correction block 104, where shading correction caused by the combination of the image sensor 100 and the main photographing optical system 118 is executed, or sensor-specific pattern noise is removed. To make corrections.

  As described above, the output of the correction block 104 is sequentially stored as frame data in the buffer memory 106 via the front memory controller 105, and an image at the time of continuous shooting by the camera is temporarily stored. Become.

  As shown in the figure, the front memory controller 105 is synchronized with the image sensor 100 by operating based on a signal from a timing control block 107 that operates in synchronization with the timing generator 102.

  When shooting of at least one frame is completed, the data in the buffer memory 106 storing the shooting data is temporarily transferred to the work memory 111 via the rear memory controller 108 under the control of the front memory controller 105. To do.

  The rear memory controller 108 operates based on a signal from a timing control block 109 that operates in synchronization with the timing generator 102.

  Next, the data in the work memory 111 is subjected to so-called picture creation such as color interpolation processing and matrix correction in the color processing block 112 connected to the same bus A to perform R, G, B, and Y, Cr. , Cb conversion processing is performed, and the result is stored in the work memory 111 again.

  The data stored in the work memory 111 is compressed based on a predetermined compression format via the JPEG processing block 113, and the result is sent to the card memory 115 (usually a non-volatile memory such as a flash memory via the card controller 114). To use.)

  Conversely, when observing image data that has already been photographed, the data stored in the card memory 115 is decompressed into data for each photographing pixel through the JPEG processing block 113, and the result is transferred to the work memory 111. By doing so, external display can be performed through a monitor display means (not shown).

  On the other hand, regarding the control of the entire camera, the overall control CPU 116 executes a command in accordance with the instruction code stored in the instruction memory 117 connected to the same bus B, and controls the main photographing optical system 118 via the lens control means 119. While controlling the drive, various information attached to the image via the communication I / F 110 is recorded as data in the card memory.

JP 2000-253305 A

  As described above, in the case of image processing in a normal digital camera, the continuous shot frames are stored as raw data in the buffer memory until halfway, and then sequentially transferred to the subsequent processing block to execute color processing and JPEG processing. However, in actual operation, it is possible to realize a spec of 8 frames / second or more equivalent to that of a so-called professional silver halide camera by increasing the reading speed from the image sensor. Yes.

  As a method for increasing the reading speed of the image sensor, a method of simply increasing the driving speed of the driver by increasing the frequency of the timing generator drive clock, or performing reading from the image sensor simultaneously from two or more outputs, A method has been proposed in which processing up to storage in the buffer memory is performed in parallel on a plurality of lines in accordance with the number of read outputs.

  However, as described above, after the contents of the buffer memory are expanded to the work memory via the memory controller, the data is sequentially transferred to the color processing block and then predetermined R, G, B image data or Y, Cr, When converting to Cb image data and temporarily storing it again in the work memory again, and subsequently transferring this data to the JPEG processing block, reading in block units of 8 × 8 units accompanying so-called raster block conversion Is performed via the bus A connected between the blocks shown in the figure, the bus A is occupied with considerable frequency in this processing.

  In addition, the image data actually compressed through JPEG processing is stored in the card memory. In this case, the writing speed of the card memory itself becomes a bottleneck, and naturally the result of JPEG processing is swept from the work memory. As a result, the output speed becomes slow, and as a result, a large amount of data is accumulated in the work memory.

  As a result, for the photographer himself, continuous shooting was taken unless the actual post-processing slowed down and the capacity of the buffer memory or work memory increased considerably, no matter how the camera frame speed increased. There is a problem in photographing that it is not possible to immediately shift to a later release operation.

  The present invention has been made in view of the above-described problems, and in combination with data reading from the imaging means in the imaging sequence, speeds up the processing after imaging, eliminates imaging problems, and improves reliability. An object is to provide a high imaging device and method.

The imaging apparatus of the present invention forms an image of a subject and converts the image data into an electrical signal for each pixel, and is connected in parallel to the imaging unit, and is output from the imaging unit. A plurality of image processing means for processing the electrical signals and generating image signals by processing the electrical signals, and each of the image processing means is connected to the imaging means in a switchable manner. Therefore, the image processing corresponding to each photographing frame is appropriately switched and executed.
In one aspect of the imaging apparatus of the present invention, the image processing unit temporarily stores the electrical signal output from the imaging unit, and the electrical signal stored in the temporary storage unit. Image signal generating means for reading and generating the image signal.
In one aspect of the imaging apparatus of the present invention, the imaging means has a plurality of output terminals for outputting the electrical signals to the outside, and the image processing means is connected to the output terminals.
In one aspect of the imaging apparatus of the present invention, in two of the plurality of image processing means, one of the image processing means executes image processing of odd-numbered frames, and the other of the image processing means images images of even-numbered frames. Execute the process.
In one aspect of the imaging apparatus of the present invention, the electrical signal is RAW data.
In one aspect of the imaging apparatus of the present invention, the image processing includes RAW data color interpolation processing.
In one aspect of the imaging apparatus of the present invention, each of the imaging frames is a imaging frame obtained by continuous shooting by the imaging means. The imaging method of the present invention forms a subject image, and the image data is stored in each image data. A plurality of image processing units that temporarily store the electrical signal output from the imaging unit and read the electrical signal to generate an image signal are connected in parallel to the imaging unit that converts each pixel into an electrical signal. Each image processing means can be switched using the image pickup apparatus thus configured, and the electric signals from the image pickup means are individually processed by the image processing means.
In one aspect of the imaging method of the present invention, the image processing corresponding to each photographing frame is appropriately switched and executed by each image processing means.
In one aspect of the imaging method of the present invention, in two of the plurality of image processing means, one of the image processing means executes image processing of odd-numbered shooting frames, and the other image processing means performs image processing of even-numbered shooting frames. Execute the process.
In one aspect of the imaging method of the present invention, the electrical signal is RAW data.
In one aspect of the imaging method of the present invention, the image processing includes color interpolation processing of RAW data.
In one aspect of the imaging method of the present invention, the imaging piece is obtained by continuous shooting by the imaging means.

  According to the present invention, the processing after shooting is speeded up in combination with the data reading from the imaging means in the shooting sequence, the shooting trouble is solved, and the slowdown of the normal image processing makes it possible to perform continuous shooting. The problem that the number of frames is limited is greatly improved, and a highly reliable imaging apparatus and method are realized.

  The present invention has the same function with respect to a memory controller block for capturing a photographed image as it is, a color processing block for actually making a picture, a JPEG processing block, a card memory controller, and an image processing means (image processing block) including a card memory. A configuration in which two or more devices having the above are connected and arranged in parallel to the imaging means.

Specifically, the image data from the imaging means is read through, for example, 2-channel output during continuous shooting, and the data for each channel is stored in separate buffer memories.
The output of one side of this photographic data is processed by the first image processing block, and at the same time the output of the other side is processed by the second image processing block. A method of processing is conceivable.

  In this case, each area necessary for image processing is processed with some overlap (that is, the same area on the image sensor is processed on the CH1 side and the CH2 side), and then JPEG processing is performed. Is preferably performed by one processing block by transferring data to one side. In this case, a method of switching the processing block between the odd-numbered frame and the even-numbered frame is taken.

Hereinafter, specific examples of the method and method to which the present invention is applied will be described in detail with reference to the drawings.
FIG. 1 is a block diagram illustrating a hardware configuration of the entire imaging apparatus according to the present embodiment. In the illustrated example, the overall control CPU 26 detects a change in state of a camera operation switch (not shown) (configured by a camera main switch and a release switch) by the photographer himself, and supplies power to other circuit blocks as well as the initial state. Start setting.

  The imaging device forms an image of a subject and converts the image data into an electrical signal for each pixel. The imaging device 1 processes an electrical signal output from the imaging device 1 and outputs an image signal. It has two image processing blocks 31 and 32 that are image processing means to be generated, and the image processing blocks 31 and 32 are connected to CH1 and CH2 that are output terminals of the image sensor 1, and are connected in parallel to the image sensor 1. Configured.

  The image processing block 31 is for synchronizing the memory controller 8 and the buffer memory 9 constituting temporary storage means for temporarily storing the electrical signal output from the image sensor 1 and the image sensor 1 of the memory controller 8. A timing control block 10, a JPEG processing block 14, a color processing block 13, a work memory 12, and a work memory 12 constituting image signal generating means for reading out an electrical signal stored in the temporary storage means and generating the image signal. A card memory 16 for storing the image data stored in the card controller 15 and a communication I / F 11. The memory controller 8, JPEG processing block 14, color processing block 13, work memory 12, card controller 15 And the communication I / F 11 is connected to the bus A1. .

  Similarly to the image processing block 31, the image processing block 32 also includes a memory controller 17 and a buffer memory 18 that constitute temporary storage means for temporarily storing the electrical signal output from the image sensor 1, and the image sensor of the memory controller 17. 1 and a JPEG processing block 23, a color processing block 22, and an image signal generating means for reading out an electrical signal stored in the temporary storage means and generating the image signal. A work memory 21, a card memory 25 that stores image data stored in the work memory 21 via a card controller 24, and a communication I / F 20, a memory controller 17, a JPEG processing block 23, and a color processing block 22. , Work memory 21, card controller 17 and Shin I / F20 is connected to the bus A2.

  The subject image within the photographing screen range is imaged on the image sensor (CCD in this case) 1 through the main photographing optical system 28, and the CH1 output as an output signal from the image sensor 1 is used as a CDS / AGC / AD circuit. 4 sequentially performs correlated double sampling, gain setting, and AD conversion for each pixel, and sequentially converts them into a predetermined digital signal. The other output signal from the image sensor 1 is the CH2 output, Correlated double sampling, gain setting, and AD conversion are sequentially performed for each pixel via the CDS / AGC / AD circuit 6 to sequentially convert it into a predetermined digital signal.

  The image sensor 1 is driven at a predetermined drive by the output of the driver circuit 2 for horizontal drive and vertical drive for each pixel based on a signal from a timing generator (TG / SSG) 3 that determines the overall drive timing. 1 generates an image signal output. As shown in FIG. 1, the output channel has 2CH (actually, it is possible to have a larger number of channels). As a result, the image data can be read at a higher speed than the image sensor having only a single channel as shown in FIG.

Here, a specific configuration of the image sensor used in the present embodiment will be described with reference to the configuration diagram of FIG.
FIG. 3 shows an internal configuration of an image sensor having two outputs of CH1 and CH2, wherein a is a photodiode portion that converts an actual light receiving input into a charge amount, and b is a charge generated by this photodiode. A vertical CCD unit for transferring the charge from the top to the bottom of the figure in the so-called bucket relay format, c is used to transfer the lator charge horizontally to the left and right outputs in the bucket relay format. This represents a horizontal CCD unit, and is a type that reads out each pixel symmetrically from the center of the screen.

  Each pixel indicated by the right dotted line portion 40 on the screen reads out data from the horizontal CCD portion 42 via the right output channel CH1 and inputs it to the CDS / AGC / AD 44, while each pixel indicated by the left dotted line portion 41 on the screen. Reads out data from the horizontal CCD section 43 via the left output channel CH2 and inputs the data to the CDS / AGC / AD45.

  Returning to FIG. 1 again, the output signal of the CDS / AGC / AD circuit 4 is input to the correction block 5 where the shading correction generated by the combination of the image sensor 1 and the main imaging optical system 28 is executed, or the sensor specific The correction circuit is used to correct the pattern noise and uses a built-in multiplication circuit, addition circuit, and data storage memory in the horizontal and vertical directions of the two-dimensional image data. Correction for each pixel.

  Similarly, the output signal of the CDS / AGC / AD circuit 6 is input to the correction block 7, and again here corrections such as shading correction and removal of fixed pattern noise caused by the combination of the image sensor 1 and the main imaging optical system 28 are performed. Execute.

  Subsequently, as shown in FIG. 1, the memory controller 8 is synchronized with the image sensor 1 by operating based on the signal of the timing control block 10 that operates in synchronization with the timing generator 3. As described above, the sensor signal from the imaging device 1 is sequentially converted into predetermined bus width data through the CDS / AGC / AD circuit 4 and the correction circuit block 5, and the data is transferred to the buffer memory 9 by burst (continuous) writing. .

  By the same method, the memory controller 17 synchronizes with the image sensor 1 by operating based on the signal of the timing control block 19 that operates in synchronization with the timing generator 3, and the image sensor 1 as described above. The sensor signal from is converted into predetermined bus width data through the CDS / AGC / AD circuit 6 and the correction circuit block 7 sequentially, and the data is transferred to the buffer memory 18 by burst (continuous) writing.

  As described above, the CH1 side output of the image sensor 1 (right screen of the image sensor shown in FIG. 3) is output to the buffer memory 9, and the CH2 side output of the image sensor 1 (left screen of the image sensor shown in FIG. 3) is Each of the images is recorded in the buffer memory 18, and as a result, one photographed image is stored in a memory that exists in a separate space both physically and logically.

  When the writing of the image of the predetermined size is completed, the memory controller 8 temporarily transfers the data in the buffer memory 9 storing the photographing data to the work memory 12.

  Next, the memory controller 8 sequentially transfers the data in the work memory 12 to the color processing block 13 connected to the same bus A1, and performs a so-called picture making operation.

This operation will be described with reference to the internal configuration diagram of the color processing block shown in FIG.
Image data input from the work memory 12 via the bus A is converted into a predetermined data width through the data input / output I / F 50 and then input to the color interpolation block 52. Here, first, the pixel array of the normal sensor is used. Is configured by so-called Bayer arrangement, color interpolation processing is performed to convert it into RGB three plane data.

  Subsequent to this color interpolation processing, the data is input to the matrix correction block 53, and matrix correction is performed to output a desired color from the spectral characteristics of the sensor-specific color filter, thereby converting from RGB to RGB.

  Next, a process of converting the 12-bit data width, which is normally input to the gamma conversion block 54 and converted into digital data by CDS / AGC / AD, into 8 bits is performed so as to convert the data so as to fall within a predetermined dynamic range. Perform gamma conversion.

  Subsequently, RGB to YCrCb conversion block 55 is input to perform RGB to YCrCb color conversion processing, and then input to false color removal block 56 to perform false color removal processing on the CrCb component.

  Here, the false color removal process includes using a median filter (intermediate value filter) for the occurrence of color moiré or the like caused by the relationship between so-called sampling frequency and image frequency.

  Further, the image is input to the edge emphasis block 57, so that edge emphasis processing for raising the gain near the intermediate frequency of the image is performed to perform image emphasis processing and the like, and then input to the resolution conversion block 58 to input a predetermined image size. Resize to

  Here, when resizing to a predetermined image size, the thinning process is performed after actually performing the filter process, but the same process is performed in the horizontal and vertical directions.

  The above operation is performed sequentially for one frame, and the result is developed again on another area on the work memory 12 via the data input / output I / F 50.

  The above operation is the actual operation of the actual color processing block 13 shown in FIG. 1, but the operation of each block is basically free of the setting of its characteristics via the parameter setting block 51 from the overall control CPU 26. The picture making conditions can be changed for each photographing frame.

  Subsequently, the memory controller 8 sequentially transfers the color-processed data developed in the work memory 12 to the JPEG processing block 14 connected to the same bus A1, and executes actual image compression processing here.

The operation of the JPEG processing block 14 will be described with reference to the internal configuration diagrams shown in FIGS.
FIG. 5 relates to a so-called irreversible type JPEG process, and is based on frequency conversion based on DCT conversion.

  First, data after image processing in the work memory 12 storing the result of color processing by the above-described method is read via the data input / output I / F 60 and input to the raster block conversion block 63 to obtain image data. Is converted into a block in a two-dimensional unit in units of 8 horizontal pixels and 8 vertical pixels.

  Next, data is input to the DCT conversion block 64, and here, DCT conversion is performed to convert the data into 8 × 8 data for each so-called frequency component in units of 8 × 8 blocks, and low-frequency components to high-frequency components in two-dimensional units. Calculate the coefficient to

  Next, quantization is performed on the coefficient value input to the quantization block 65 and calculated by the DCT transform. Regarding this quantization, each coefficient is based on the value of the quantization table 61 in which values are set in advance. This is realized by division every time.

  Further, the quantized result is inputted to the Huffman coding block 66 while reading out data along a predetermined scanning direction, and here, the entropy along the value of the Huffman table 62 which is also set in advance here. Encoding is performed.

  The data compressed by the above method is written back into a predetermined area of the work memory 12 through the data input / output I / F 60 again, thereby completing a series of compression processing.

On the other hand, as another type of JPEG processing, there is a reversible compression method. This method will be described with reference to the internal block diagram of FIG.
FIG. 6 relates to a reversible JPEG process based on the so-called DPCM. First, the data after image processing in the work memory 12 storing the result of color processing by the above-described method is input / output. The image data is read via the I / F 70 and input to the DPCM conversion block 72 to convert the image data as difference data from the predicted value.

  Next, the DPCM-converted data is input to the Huffman encoding block 73 while being read, and entropy encoding is performed according to the value of the Huffman table 71 in which values are set in advance.

  The data compressed by the above method is written back into another predetermined area of the work memory 12 through the data input / output I / F 70 again, thereby completing a series of compression processing.

  After compressing data based on a predetermined compression format via the JPEG processing block 14 by the above method, the compressed data is transferred to the card memory 16 (usually using a non-volatile memory such as a flash memory) via the card controller 15. Remember.

  On the other hand, when observing captured image data, the data stored in the card memory 16 is stored in the JPEG processing block 14 (however, in the JPEG processing block configuration diagrams shown in FIGS. The block that decompresses the data is not shown.) The data is expanded to the data for each normal photographing pixel, and the result is transferred to the work memory 12, so that the photographed image is reduced through the monitor display means (not shown). Can be displayed externally.

  The above is a description of the configuration of the image processing block 31 shown by the broken line in FIG. 1, but the image processing block 32 that processes data output via the memory controller 6 is similarly shown by the broken line. ing.

  In the image processing block 32, the data from the correction block 7 is stored in advance in the buffer memory 18 via the memory controller 17, and the captured image is transferred to the work memory 21 via the memory controller 17.

  Further, the memory controller 17 reads out the data in the work memory 21 based on the timing signal from the timing control block 19 and transfers it to the color processing block 22, where the actual picture making process is performed according to the method described above. The result is transferred to the work memory 21 again.

  Subsequently, the memory controller 17 reads out the image-processed data stored in the work memory 21 and transfers it to the JPEG processing block 23. Here, the memory controller 17 performs JPEG compression processing according to the method described above, and converts the compressed data into the work data again. The data is written back to the memory 21.

  Further, the compressed data is written into the card memory 25 via the card controller 24, and the recording of the photographed image is completed.

  On the other hand, with respect to the control of the entire camera and the sequence control for the controller, the overall control CPU 26 operates by executing an instruction according to the instruction code stored in the instruction memory 27 connected to the bus B, and the lens control means. 29, the main photographing optical system 28 is driven and controlled (focus drive and aperture drive control in the lens), or actual shutter exposure control is performed via a shutter control unit (not shown), and an image is transmitted via the communication I / F 11. Header information is added to the photographed image in the processing block 31, or information such as photographing conditions is added as data in the card memory and recorded.

  Similarly, the same processing as described above is performed on the image processing block 32 via the communication I / F 20, and various information is added and recorded.

  The above is a description of the configuration of the entire block shown in FIG. 1. As shown in the above configuration, the right side of the screen of the image sensor 1 is processed by the image processing block 31, and the left side of the screen is processed by the image processing block 32. Therefore, in actuality, regarding the filter processing at the color processing, it is necessary to process the area in the center of the screen while overlapping both of the image processing blocks 31 and 32.

  Also, with regard to JPEG processing, it is not practical to divide and process one photographic frame into left and right. Actually, when color processing is completed, data is transferred to one of the image processing blocks 31 and 32. Thus, a method of performing compression processing using one JPEG processing block is adopted.

  In this case, some kind of data transfer is required between the image processing block 31 and the image processing block 32. With respect to the data transfer method between the image processing blocks 31 and 32, the shooting sequence timing diagram shown in FIG. 2 is used. Give an explanation.

FIG. 2 simply shows the timing when continuous shooting or the like is performed with an actual camera.
Here, the shutter exposure timing is shown at the top, and in this case, as a continuous shooting operation, shooting is performed at approximately the same interval, and shooting is performed up to the 10th frame.

  Below that, the sensor readout timing is shown. The sensor readout is performed simultaneously with the completion of the shutter exposure for each frame, and the captured image data is stored in the buffer memory 9 under the control of the memory controller 8 in the image processing block 31 as described above. I am writing.

  Similarly, the eighth timing from the top indicates the sensor readout timing for the image processing block 32. The photographing data is stored in the buffer memory 18 in accordance with the control of the memory controller 17 in the same manner as the timing of the image processing block 31. I am writing.

  Next, image processing is started with respect to the first photographed image. In this case, the memory controller is in accordance with an instruction from the overall control CPU 26 as in the communication I / F timing shown at the seventh position from the top. 8 starts the operation, and the stored image in the buffer memory 9 is transferred to the work memory 12 as shown at the third timing from the top in FIG.

  Similarly, for the image processing block 32, as shown at the bottom timing in FIG. 2, the memory controller 17 starts operating in response to an instruction from the overall control CPU 26, and is shown fifth from the bottom in FIG. The recorded image in the buffer memory 18 is transferred to the work memory 21 at the same timing.

The fourth from the top shows the operation timing of the color processing in the image processing block 31, and the lower one shows the operation timing of the JPEG processing in the image processing block 31.
Similarly, the fourth from the bottom shows the operation timing of color processing in the image processing block 32, and the lower shows the operation timing of JPEG processing in the image processing block 32.

  Here, when color processing is executed in each block, when image processing is performed on the premise of the image sensor as shown in FIG. 3, the vicinity of the boundary at the center of the screen overlaps each other (schematically shown in FIG. 7). As shown in FIG. 5, the predetermined range with the screen center as a boundary needs to be processed on both the left side of the screen and the image processing block 31 side, and the right side of the screen is also processed on the image processing block 32 side).

Therefore, in this case, as shown in the timing of the communication I / F in FIG. 2, the operation of transferring the image data to the other party is performed.
For example, at the communication I / F timing shown at the seventh position from the top in FIG. 2, a part of the image stored in the work memory 21 is transferred to the work memory 12 as the transfer between work memories. At the communication I / F timing shown at the bottom, a method of transferring a part of the image stored in the work memory 12 to the work memory 21 as the transfer between work memories is adopted.

  As for JPEG processing, after transferring the image data on the work memory storing the result processed by the color processing block from one block to one block via the communication I / F, It is assumed that the compression operation is performed in one JPEG processing block.

  As shown in the communication I / F timing, the contents of the image data storing the result of the color processing on the image processing block 32 side for the first frame of shooting data are the work memory 12 of the image processing block 31. Then, the entire image of the first frame is processed by the JPEG processing block 14.

  Furthermore, as shown at the sixth timing from the top, the compressed data stored in the work memory 12 is written into the card memory 16 via the card controller 15.

  Finally, when the card writing operation for the first frame in the image processing block 31 is completed, a completion interrupt signal from the memory controller 8 is transmitted to the overall control CPU 26 to notify that the processing for the first frame is completed.

  Next, in the case of processing for the second frame, the memory controller 8 in the image processing block 31 and the memory controller 17 in the image processing block 32 via the communication I / Fs 11 and 20 as described above. The operation is started, and similar color processing and JPEG processing are simultaneously started in each block.

  However, in this case, the color processing result is transferred from the image processing block 31 to the work memory of the image processing block 32 via the communication I / Fs 11 and 20, and then the JPEG processing block 23 in the image processing block 32. A compression process is executed, and data is written to the card memory 25 via the card controller 24.

Finally, when the first card writing operation in the image processing block 32 is completed, a completion interrupt signal from the memory controller 17 is transmitted to the overall control CPU 26 to notify the completion of the second frame processing. .
In actual operation, before the image compression process for the first frame in the image processing block 31 is completed, the image compression process for the second frame is started, so the first frame and the second frame are captured. Are stored in separate card memories.

  At the timing shown in this figure, the processing for odd-numbered frames is processed by the image processing block 31, and the processing for even-numbered frames is processed by the image processing block 32. However, the processing is not necessarily limited to such frames. Another possible method is to use a block whose operation has been completed before starting the processing.

  For example, when the processing of the second frame is completed in the image processing block 32 before the processing of the first frame is completed in the image processing block 31 (the first frame and the second frame have a time difference in this timing diagram, As the processing proceeds, the timing at which the image processing block 31 and the image processing block 32 start processing approaches, and the processing time may be reversed.) In the image processing block 32, the third frame processing is continued. It is also possible to do this.

  This is because the overall control CPU 26 detects a process completion interrupt from the memory controller in each block via each communication I / F block, thereby determining in which block the process for the next photographing frame is executed. Has been realized.

  In the present embodiment, the case where the two image processing blocks 31 and 32 are connected in parallel to the image sensor 1 is illustrated, but the present invention is not limited to this configuration, and three or more image processing blocks are used. May be connected in parallel so as to be switched as appropriate in accordance with the photographing situation or the like.

  As described above, according to the present embodiment, a high-reliability imaging apparatus that speeds up post-shooting processing in combination with data reading from the imaging means in the shooting sequence, eliminates shooting problems, and is highly reliable. Is realized. That is, during high-speed continuous shooting of the camera, image data from the image sensor is read out at a high speed via a plurality of line outputs, and temporarily passed through a controller connected to each line in the form (RAW data format). While continuously storing in each buffer memory, the processing for the first frame of the captured image is divided into separate blocks at the same time, while the processing for the second frame of the captured image also starts partly. By adopting this method, the problem that the number of frames during continuous shooting is limited due to the slowness of normal image processing is greatly improved.

  Further, when such multi-processing is executed, a fixed configuration in which processing for odd frames is simply performed by the first image processing block and processing for even frames is performed by the second image processing block. Instead, more advanced functions are used, such as preferentially using image processing blocks that have already completed image processing and are ready to start the next processing. By configuring this multi-processing system, a higher-speed image processing system can be realized.

It is a block diagram which shows the hardware constitutions of the whole imaging device which concerns on this embodiment. It is a timing diagram showing the actual operation timing concerning this embodiment. It is a specific block diagram of the image pick-up element used for this embodiment. It is a block diagram which shows the structure of the image processing which concerns on this embodiment. It is a block diagram which shows the structure of the irreversible type compression process which concerns on this embodiment. It is a block diagram which shows the structure of the reversible type compression process which concerns on this embodiment. It is a conceptual diagram which shows the image processing range in the image pick-up element used for this embodiment. It is a block diagram which shows the hardware constitutions of the conventional imaging device whole.

Explanation of symbols

1 Image sensor 2 Driver circuit 3 TG / SSG
4, 6 CDS / AGC / AD circuit 5, 7 Correction block 8, 17 Memory controller 9, 18 Buffer memory 10, 19 Timing control block 11, 20 Communication I / F
12, 21 Work memory 13, 22 Color processing block 14, 23 JPEG processing block 15, 24 Card controller 16, 25 Card memory 26 Overall control CPU
27 Instruction memory 28 Main photographing optical system 29 Lens control means 31, 32 Image processing block

Claims (13)

Imaging means for forming a subject image and converting the image data into an electrical signal for each pixel;
Wherein are connected in parallel to the imaging means, the electric signal output from the imaging means is freely respectively individually processed, a plurality of image processing means for generating an image signal by processing said electrical signal Including
Each of the image processing means is connected to the imaging means so as to be switchable, and appropriately executes image processing corresponding to each photographing frame . The image processing means includes
Temporary storage means for temporarily storing the electrical signal output from the imaging means;
The image pickup apparatus according to claim 1, further comprising: an image signal generation unit that reads out the electrical signal stored in the temporary storage unit and generates the image signal.   3. The image pickup means has a plurality of output terminals for outputting the electric signal to the outside, and the image processing means is connected to each of the output terminals. Imaging device. 2. Two of the plurality of image processing means, wherein one of the image processing means performs image processing for odd-numbered frames, and the other of the image processing means performs image processing for even-numbered frames. Item 4. The imaging device according to any one of Items 1 to 3 . The imaging apparatus according to claim 1, wherein the electrical signal is RAW data. The imaging apparatus according to claim 5, wherein the image processing includes color interpolation processing of RAW data. The imaging apparatus according to claim 1, wherein each of the shooting frames is a shooting frame obtained by continuous shooting by the imaging unit. The electrical signal output from the imaging unit is temporarily stored in an imaging unit that forms a subject image and the image data is converted into an electrical signal for each pixel, and the electrical signal is read out to obtain an image signal. Using an imaging device in which a plurality of image processing means to be generated are connected in parallel,
An image pickup method characterized in that each image processing means is switchable, and each electric signal from the image pickup means is individually processed by each image processing means. The imaging method according to claim 8, wherein the image processing corresponding to each imaging frame is appropriately switched and executed by each image processing unit. 2. Two of the plurality of image processing means, wherein one of the image processing means performs image processing for odd-numbered frames, and the other of the image processing means performs image processing for even-numbered frames. Item 10. The imaging method according to Item 9. The imaging method according to claim 8, wherein the electrical signal is RAW data. The imaging method according to claim 11, wherein the image processing includes color interpolation processing of RAW data. The imaging method according to claim 9 or 10, wherein each of the imaging frames is an imaging frame obtained by continuous shooting by the imaging means.

Images