JP2008124671A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a plurality of moving image data with different processing parameters of a camera signal processing circuit. <P>SOLUTION: In high speed imaging mode, 240fps RAW data from an image sensor 101 are supplied to a camera signal processing circuit 103. Parameters of the camera signal processing circuit 103 are switched in synchronism with frame switching of 240fps RAW data. Each 60fps image is processed by a different parameter, and processed image data are compressed by a compression/decompression circuit 105 and are written into a memory 107. In writing or reading the image data, 4 moving image data are formed from the processed image data. Selected moving image data are read from the memory 107 and are decompressed by the compression/decompression circuit 105, and displayed on a display part 112 via a conversion processing part 104 and a display processing circuit 108. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus and an image pickup method, and more particularly to an image pickup apparatus and an image pickup method capable of obtaining a plurality of moving image pickup signals with different shooting conditions during moving image shooting.

  2. Description of the Related Art Conventionally, in a digital still camera, there is known an auto bracket function (hereinafter referred to as bracket shooting) that sequentially captures a plurality of shot images to which different shooting conditions are sequentially applied during shooting. For example, three continuous shots are automatically made with the exposure shifted. Each image photographed by continuous shooting is stored in a memory, and the most preferable one is selected from the three images after photographing. Japanese Patent Application Laid-Open Publication No. 2004-228561 describes a technique that allows a user to switch between bracket shooting and an imaging mode in which imaging conditions are automatically set.

JP 2006-050336 A

  Patent Document 2 below describes an imaging apparatus that can individually set imaging conditions for each of a plurality of images obtained by bracket imaging.

JP 2006-254323 A

  The above-described bracket shooting of the imaging apparatus proposed above is applied to still image shooting and not to moving image shooting. Since the conventional imaging device outputs 60 captured images per second, when performing four types of moving image capturing, the image rate is 15 images per second, and image degradation due to motion occurs. Therefore, when it is desired to perform bracket shooting for a moving image in a conventional imaging device, it is necessary to change the imaging condition and perform shooting again after shooting under a specific imaging condition. This method has a problem that it is troublesome to repeat photographing a plurality of times, and the same captured image cannot be obtained when the subject moves.

  Accordingly, an object of the present invention is to provide an imaging apparatus and an imaging method capable of performing bracket shooting for a moving image.

In order to solve the above-described problem, the present invention provides camera signal processing means for performing image processing on imaging data of a first screen rate obtained from a solid-state imaging device,
First compression encoding means for compressing image data of a first screen rate processed by the camera signal processing means;
Temporary storage means for temporarily storing image data encoded by the first compression encoding means;
Decoding means for reading image data captured during a predetermined time from a temporary storage means at a second screen rate lower than the first screen rate, and decompressing the read image data;
Second compression encoding means for compressing the image data expanded by the decoding means;
Storage means for storing image data encoded by the second compression encoding means;
And a control unit that controls the camera signal processing unit so that the processing parameter is switched in a time division manner for each of one or a plurality of images.

The present invention includes a camera signal processing step for performing image processing on imaging data of a first screen rate obtained from a solid-state imaging device;
A first compression encoding step for compressing the image data of the first screen rate processed by the camera signal processing step;
A temporary storage step of temporarily storing the image data encoded by the first compression encoding step;
A decoding step of reading image data captured during a predetermined time from a temporary storage step at a second screen rate lower than the first screen rate, and decompressing the read image data;
A second compression encoding step for compressing the image data expanded by the decoding step;
A storage step of storing the image data encoded by the second compression encoding step;
And a control step for controlling the camera signal processing step so that the processing parameter is switched in a time-sharing manner for each of one or a plurality of images.

According to the present invention, recording medium control means for recording imaging data of a first screen rate obtained from a solid-state imaging device on a recording medium and reading the imaging data from the recording medium at a second screen rate lower than the first screen rate. When,
Conversion means for converting imaging data of the first screen rate from the solid-state imaging device into imaging data of the second screen rate;
Camera signal processing means for performing camera signal processing on imaging data at the second screen rate from the conversion means or the recording medium control means;
Display processing means for generating an image signal for display from the image signal from the camera signal processing means;
And a control unit that controls the solid-state image sensor so that the parameters of the solid-state image sensor are switched in a time division manner for each of one or a plurality of images.

This invention records the imaging data of the 1st screen rate obtained from the solid-state image sensor on a recording medium, and the recording medium control step which reads imaging data from the recording medium with the 2nd screen rate lower than the 1st screen rate When,
A conversion step of converting imaging data of a first screen rate from a solid-state imaging device into imaging data of a second screen rate;
A camera signal processing step for performing camera signal processing on the imaging data of the second screen rate from the conversion step or the recording medium control step;
A display processing step for generating an image signal for display from the image signal from the camera signal processing step;
And a control step for controlling the solid-state imaging device so that the parameters of the solid-state imaging device are switched in a time division manner for each of one or more images.

  According to the imaging apparatus of the present invention, a plurality of moving image data having different processing parameters for camera signal processing can be obtained by one imaging, and desired moving image data can be determined immediately after imaging. In the present invention, the screen rate of a plurality of moving image data is, for example, a normal 60 fps, and there is an advantage that image quality deterioration does not occur when displayed.

  Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. As illustrated in FIG. 1, an imaging signal from an image sensor 101 is supplied to the imaging apparatus 100 according to the first embodiment. The imaging apparatus 100 includes an image sensor 101, a preprocessing circuit 102, a camera signal processing circuit 103, a conversion processing unit 104, a compression / decompression circuit 105 serving as a first compression encoding / decoding unit, a memory control circuit 106, and a temporary storage unit. A memory 107, a display processing circuit 108, a compression / decompression circuit 109 as a second compression encoding / decoding unit, a recording device control circuit 110, a recording device 111 as a storage unit, a display unit 112, and a control unit 113. I have.

  The image sensor 101 converts incident light from a subject captured via an optical system (including a lens, an infrared ray removal filter, an optical low-pass filter, and the like) into an electrical signal by photoelectric conversion. For example, a complementary metal oxide semiconductor (CMOS) type image sensor is used. In the case of a COMS type image pickup device, photodiodes, row / column selection MOS transistors, signal lines, and the like are two-dimensionally arranged to form a vertical scanning circuit, a horizontal scanning circuit, a noise removal circuit, a timing generation circuit, and the like. As the image sensor 101, a CCD (Charge Coupled Device) capable of high-speed imaging may be used.

  The image sensor 101 has a high-speed imaging mode in which signals are read at a high speed at a first screen rate (also referred to as a frame rate) equal to or higher than a normal screen rate (60 fps (field / second)), which is a specification of the NTSC system, The normal imaging mode for reading out signals at the second screen rate can be switched. The screen rate in the high-speed imaging mode is 240 fps, which is N times the normal rate (N is an integer of 2 or more), for example, 4 times.

  The image sensor 101 includes an analog front end such as CDS (Correlated Double Sampling) and an A / D converter, and outputs a digital imaging signal corresponding to the pixel arrangement of the image sensor 101. The digital imaging signal output from the image sensor 101 is data that has not been processed by the preprocessing circuit 102 and the camera signal processing circuit 103, and is referred to as RAW data in this specification.

  The image sensor 101 uses, for example, three image sensors that output image signals of three primary colors, and outputs one row every four rows from each image sensor, so that 4 at a normal screen rate (60 fps) is obtained. Double the screen rate of 240 fps can be obtained. If the number of pixels in one frame at a normal screen rate is, for example, 6.4 million pixels, it is 1.6 million pixels in the high-speed imaging mode.

  FIG. 2A shows an example of an arrangement of color filters to which the present invention can be applied. The square grid array is tilted by 45 °, and the entire periphery of each of the R and B filters is surrounded by a G filter. With this configuration, necessary and sufficient spatial frequency characteristics can be obtained for the human visual sensitivity characteristics for the R and B components, but the spatial frequency characteristics of the G component, which has a higher human sensitivity than these components, can be achieved compared to the conventional Bayer array. It becomes possible to raise. The G component is a main component in generating a luminance signal, and this improves the resolution of luminance for not only an achromatic subject but also a chromatic subject, thereby improving the image quality. In the present invention, an image sensor having various color filter arrangements including a general square lattice arrangement other than the color filter arrangement shown in FIG. 2A may be used.

  In the color filter array shown in FIG. 2A, when the screen rate is normal, a method of alternately reading out pixels in two adjacent rows within one horizontal period as shown by a broken line is basic. That is, in the normal imaging mode, reading is performed by sequentially scanning in this way.

  In the case of a high screen rate, reading is performed in which scanning lines are thinned out at a rate of one for four horizontal scanning lines. In order to correspond to the high screen rate, as shown in FIG. 2B, when the readout outputs of the adjacent six columns of pixels are CH1 to CH6, three columns (CH1, CH3, CH5, and CH2) arranged one by one are arranged. , CH4, CH6), an A / D converter is arranged in common, and one sample is converted into 14-bit digital data by the A / D converter. You may make it arrange | position an A / D converter for every row | line | column of each pixel. Further, although not shown, a CDS is arranged similarly to the A / D converter, and high-speed reading is possible. As the image sensor 101, another configuration, for example, a configuration in which filters of three primary colors are arranged for one image sensor can be used. Furthermore, an image sensor using a complementary color filter can also be used.

  The preprocessing circuit 102 performs optical correction processing such as shading correction on the RAW data output from the image sensor 101 and outputs a digital image signal. The camera signal processing circuit 103 performs camera signal processing such as white balance adjustment processing (also referred to as processing processing, development processing, picture making processing, etc.) on the imaging data from the preprocessing circuit 102.

  The conversion processing unit 104 performs display thinning and size adjustment for converting the image signal received from the camera signal processing circuit 103 into a screen rate and a screen size suitable for display on the display unit 112. Note that display thinning is performed only when output to the display processing circuit 108. Here, the display thinning is to thin out the fields so as to match the number of fields per unit time (60 fps in this case) defined by the display standard of the display device that is projected when the imaging device 100 is in the high-speed imaging mode. .

  The compression / decompression circuit 105 performs compression coding processing on the captured image data from the conversion processing unit 104 using a still image coding method such as JPEG (Joint Photographic Experts Group). Further, decompression decoding processing is performed on the encoded data of the still image supplied from the memory control circuit 106. The memory control circuit 106 controls writing and reading of image data with respect to the memory 107. The memory 107 is a FIFO (First In First Out) type buffer memory that temporarily stores the image data received from the memory control circuit 106. For example, an SDRAM (Synchronous Dynamic Random Access Memory) is used.

  The display processing circuit 108 generates an image signal to be displayed on the display unit 112 from the image signal received from the conversion processing unit 104 or the compression / decompression circuit 109, and sends the signal to the display unit 112 to display the image. . The display unit 112 includes, for example, an LCD (Liquid Crystal Display), and displays a camera-through image being captured, an image obtained by reproducing data recorded on the recording device 111, and the like.

  The compression / decompression circuit 109 performs compression encoding processing on the image data received from the conversion processing unit 104 by a moving image encoding method such as MPEG (Moving Picture Experts Group). Further, decompression decoding processing is performed on the encoded data of the moving image supplied from the recording device 111 and output to the display processing circuit 108. The display unit 112 displays the moving image received from the display processing circuit 108.

The control unit 113 includes, for example, a CPU (Central Processing Unit) and a ROM (Read Only).
Memory), RAM (Random Access Memory), and the like. The microcomputer controls the respective units of the image pickup apparatus by executing a program stored in a ROM or the like. Usually, the control unit 113 is provided as a component of part of the imaging apparatus 100.

  As the recording device 111 for storing MPEG encoded image data, a magnetic tape, a semiconductor memory such as a flash memory, a recordable optical disk, a hard disk, or the like can be used. As the recording device 111, basically, a device that can be attached and detached is applied. However, the recorded data may be output to the outside via a communication interface as being not attachable / detachable.

  In the imaging apparatus of FIG. 1, in the high-speed imaging mode, 240 fps RAW data from the image sensor 101 is supplied to the camera signal processing circuit 103 via the preprocessing circuit 102. In the camera signal processing circuit 103, as will be described later, the camera signal processing parameter is changed for each of the first, second, third, and fourth RAW data of 60 fps. The output signal of the camera signal processing circuit 103 is thinned by a quarter so that the conversion processing unit 104 becomes 60 fps. The image signal subjected to the thinning and size conversion is supplied to the display processing circuit 108, and the display processing circuit 108 generates an image signal to be displayed on the display unit 112, and sends this signal to the display unit 112 to display the image. Let

  When a recording request for an image captured at high speed is received from the control unit 113 in response to a user operation, the conversion processing unit 104 also sends a 240 fps image signal to the compression / decompression circuit 105. The conversion processing unit 104 reduces the image signal received from the camera signal processing circuit 103 as necessary, and sends the image signal to the compression / decompression circuit 105.

  The compression / decompression circuit 105 compresses and encodes the image signal from the conversion processing unit 104 in the JPEG format. The memory control circuit 106 temporarily stores the encoded data from the compression / decompression circuit 105 in the memory 107. In this way, image data for a certain time is accumulated in the memory 107.

  When the image data stored in the memory 107 is full, the imaging and recording in the high-speed imaging mode are automatically stopped, and the mode is switched to the mode in which data reproduction and recording to the recording device 111 are performed. That is, when the memory control circuit 106 receives a read request for the encoded data temporarily stored in the memory 107 from the control unit 113, the memory control circuit 106 reads the encoded data stored in the memory 107 at 60 fps and sends it to the compression / decompression circuit 105. At this time, the image signal is not thinned out. The compression / decompression circuit 105 decompresses and decodes the encoded data from the memory control circuit 106 and sends the decoded data to the conversion processing unit 104.

  Upon receiving a recording request from the control unit 113 to the recording device 111, the conversion processing unit 104 sends the image signal received from the compression / decompression circuit 105 to the compression / decompression circuit 109. The compression / decompression circuit 109 compresses the image signal received from the conversion processing unit 104 in the MPEG format, and stores the compression-coded signal in the recording device 111 via the recording device control circuit 110. The conversion processing unit 104 adjusts the size of the image signal received from the compression / decompression circuit 105 at 60 fps, sends it to the display processing circuit 108, and the reproduced image is displayed on the display unit 112.

  The configuration of an example of the preprocessing circuit 102 and the camera signal processing circuit 103 is shown in FIG. Raw data from the image sensor 101 (including CDS and A / D converter) is supplied to the shading correction circuit 231 of the preprocessing circuit 102. The shading correction circuit 231 corrects the brightness around the screen from becoming dark. An output signal of the shading correction circuit 231 is supplied to the camera signal processing circuit 103.

  As an example, the camera signal processing circuit 103 includes a gain correction circuit 240, a synchronization circuit 241, a white balance correction unit 242, an aperture correction unit 243, a gamma correction unit 244, and a YC generation unit 245 in order from the input side. ing. An image signal composed of a luminance signal (Y) and a color difference signal (C) is output to the output of the YC generation unit 245. However, the configuration of the camera signal processing circuit 203 is not limited to that shown in FIG. 3. For example, the order of these components can be changed, or some components can be omitted.

  The gain correction circuit 240 corrects the gain of the RAW data. When the gain of the gain correction circuit 240 is increased, the amplitude of the RAW data increases and the captured image becomes bright. When the gain is decreased, the amplitude of the RAW data decreases and the captured image becomes dark. .

  The synchronization circuit 241 interpolates missing pixels for each color component. Three primary color signals (R, G, B) are output from the synchronization circuit 241 in parallel. An output signal of the synchronization circuit 241 is supplied to the white balance correction unit 242. The white balance correction unit 242 corrects an unbalance between colors due to a difference in the color temperature environment of the subject and a difference in sensitivity due to a color filter on the sensor.

  The output of the white balance correction unit 242 is supplied to the aperture correction unit 243. The aperture correction unit 243 is contour correction that extracts and emphasizes a portion with a large signal change. An output signal of the aperture correction unit 243 is supplied to the gamma correction unit 244.

  The gamma correction unit 244 corrects input / output characteristics so that gradation is correctly reproduced when an imaging signal is output to the display unit 112. An output signal of the gamma correction unit 244 is supplied to the YC generation unit 245.

  The YC generation unit 245 generates a luminance signal (Y) and a color difference signal (C). The luminance signal is generated by synthesizing the gamma-corrected RGB signal at a predetermined synthesis ratio. The color difference signal is generated by combining the gamma-corrected RGB signal at a predetermined combining ratio. The generated luminance signal and color difference signal are supplied to the display unit 112 via the display processing circuit 108.

  The gain of the gain correction circuit 240, the coefficient for determining the balance between the colors of the white balance correction unit 242, the degree of contour enhancement of the aperture correction unit 243, and the input / output characteristics of the gamma correction unit 244 are used for image processing of the camera signal processing circuit 103. It is a parameter. The processing parameters of the camera signal processing circuit 103 can be set for each of N (N = 4 in this example) images. However, it is not necessary to control the parameters of all these components, and at least one machining parameter is controlled. The control of the parameters is controlled by the control unit 113 in synchronization with the RAW data.

  FIG. 4 shows control of parameters of the camera signal processing circuit 103. As indicated by reference numeral 121, the parameters of the camera signal processing circuit 103 are switched by the control of the control unit 113, and parameters A, B, which are time-division multiplexed in four phases in synchronization with frame switching of 240 fps RAW data, Parameter C and parameter D are generated. The parameters A to D are gains of the gain correction circuit 240, for example. In practice, a gain instruction signal for specifying parameters A to D is supplied to the gain correction circuit 240, and the gain correction circuit 240 is controlled to have a gain corresponding to the gain instruction signal. Processing of raw data 122 (60 fps images A, B, C, and D are repeated) input to the camera signal processing circuit 103, for example, gain correction is performed by corresponding parameters A to D, respectively.

  These parameters A to D are set in advance by the user. One of the parameters A to D, for example, the parameter A is a standard value. The standard value is an optimum value that is automatically set by the control unit 113 of the imaging apparatus 100 according to the level of the image signal. The other parameters B, C and D are set to different values. For example, the parameter B is a signal indicating a gain that is 2/3 of the gain indicated by the parameter A, and the parameter C is a signal indicating a gain that is 5/4 of the gain indicated by the parameter A. D is a signal indicating a gain that is 3/2 of the gain indicated by the parameter A.

  The parameters A to D do not need to control the same control target, the parameter A is a standard value of all parameters, and the parameter B is a value obtained by changing only the gain from the standard value. The parameter C may be obtained by changing only the white balance correction with respect to the standard value, and the parameter D may be obtained by changing only the aperture correction with respect to the standard value. When the user sets the parameters A to D to desired values, for example, a parameter setting screen can be displayed on the display unit 112, and the user can set the parameters to a desired value while viewing the screen. Further, a plurality of parameter sets in which at least two values of the parameters A to D are combined in advance may be prepared, and a desired parameter set among them may be selectively applied to the RAW data. For example, a plurality of imaging modes can be set according to the subject, shooting environment, etc., and a parameter set is selected corresponding to the imaging mode.

  Each of the 60 fps images A to D is processed by the parameters A to D, and the processed image data 123 is generated from the camera signal processing circuit 103. For example, image A is processed by parameter A to obtain image data (Y11, Cr11, Cb11), image B is processed by parameter B to obtain image data (Y12, Cr12, Cb12), and image C is parameter C. Is processed to obtain image data (Y13, Cr13, Cb13), and image D is processed by parameter D to obtain image data (Y14, Cr14, Cb14). Next, image data (Y21, Cr21, Cb21) (Y22, Cr22, Cb22) (Y23, Cr23, Cb23) (Y24, Cr24, Cb24) similarly processed by the parameters A to D are obtained.

  The image data processed by the camera signal processing circuit 103 is compressed by the compression / decompression circuit 105 and written to the memory 107 under the control of the memory control circuit 106. When writing image data to the memory 107 or reading image data from the memory 107, four moving image data 124A, 124B, 124C, and 124D are formed from the processed image data by address control.

  Of the moving image data 124A to 124D, for example, moving image data 124A is read from the memory 107 at 60 fps, expanded by the compression / decompression circuit 105, and displayed on the display unit 112 via the conversion processing unit 104 and the display processing circuit 108. On the display unit 112, a moving image processed with the parameter A is displayed. Similarly, the moving image data 124B, 124C, and 124D processed by the parameters B, C, and D are displayed on the display unit 112, respectively.

  In this manner, four moving image data processed with different parameters can be stored in the memory 107, and four moving images processed with different parameters can be sequentially displayed on the display unit 112. By looking at the displayed moving image, it is possible to determine the most preferable moving image and select the moving image data. When four moving images are displayed, four moving images may be simultaneously displayed on each of four screens obtained by reducing an image and dividing one screen.

  The selected moving image data is compressed by the compression / decompression circuit 109 and recorded on the recording device 111 via the recording device control circuit 110. The recorded moving image data is 60 fps. For example, when the recording device 111 is detached and reproduced by another reproducing device, a reproduced image can be displayed on a normal display device. If the recording device 111 has a sufficient capacity, two or more different moving image data may be recorded on the recording device 111.

  Next, another embodiment of the present invention will be described. As illustrated in FIG. 5, an imaging signal from an image sensor 101 is supplied to an imaging apparatus 200 according to another embodiment.

  The image sensor 101 is the same as the image sensor in the embodiment. That is, the image sensor 101 reads a signal at a high speed at a first screen rate (also referred to as a frame rate) equal to or higher than a normal screen rate (60 fps (field / second)), which is a specification of the NTSC system, It is possible to switch between a normal imaging mode in which signals are read out at a normal second screen rate. The screen rate in the high-speed imaging mode is 240 fps, which is four times the normal rate, for example. The image sensor 101 incorporates an analog front end such as a CDS (Correlated Double Sampling) and an A / D converter, and outputs a digital imaging signal corresponding to the pixel arrangement of the image sensor 101.

  As described above, the image sensor 101 has a configuration using three image sensors, an arrangement of color filters as shown in FIG. 2, and a configuration including an analog front end and an A / D converter. Have. In addition, an image sensor having an arrangement other than the arrangement of the color filters shown in FIG. 2 can be used.

  The imaging apparatus 200 includes a conversion processing unit 201, a preprocessing circuit 202, a camera signal processing circuit 203, a display processing circuit 208, a recording device control circuit 210, a recording device 111, a display unit 112, and a control unit 213.

  The conversion processing unit 201 performs signal diversion and display thinning on the RAW data received from the image sensor 101. Note that display thinning is performed only when output to the display processing circuit 208. Here, the display thinning is to thin out the fields so as to match the number of fields per unit time (60 fps in this case) stipulated in the display standard of the display device that is projected when the imaging device 200 is in the high-speed imaging mode. .

  The preprocessing circuit 202 performs optical correction processing such as shading correction on the digital image signal output from the image sensor 101 and outputs the digital image signal. The camera signal processing circuit 203 performs camera signal processing (also referred to as processing processing, development processing, picture making processing, etc.) such as white balance adjustment processing on the image signal from the preprocessing circuit 202. An output signal of the camera signal processing circuit 203 is supplied to the display processing circuit 208.

  The display processing circuit 208 generates an image signal to be displayed on the display unit 112 from the image signal received from the camera signal processing circuit 203, and sends this signal to the display unit 112 to display an image. The display unit 112 includes, for example, an LCD (Liquid Crystal Display), and displays a camera-through image being captured, an image obtained by reproducing data recorded on the recording device 111, and the like. Note that the display unit 112 may be provided outside the imaging device 200, and an interface for external output may be provided in the imaging device 200 instead of the display unit 112.

  A recording device control circuit 210 connected to the conversion processing unit 201 controls writing and reading of image data to and from the recording device 111. Data stored in the recording device 111 is RAW data that has not been processed by the preprocessing circuit 202 and the camera signal processing circuit 203.

  As the recording device 111, a magnetic tape, a semiconductor memory such as a flash memory, a hard disk, or the like can be used. As the recording device 111, a device that is basically not removable is used. However, the RAW data may be taken out externally by making the recording device 111 removable. When the RAW data is extracted to the outside, the RAW data processed by the preprocessing circuit 202 is preferably extracted. Camera signal processing (image processing) is performed by software of an external personal computer, for example.

For example, the control unit 213 includes a CPU (Central Processing Unit) and a ROM (Read Only).
This is a microcomputer composed of a memory (RAM), a random access memory (RAM), and the like, and comprehensively controls each unit of the imaging apparatus by executing a program stored in a ROM or the like. The control unit 213 is provided as a component of part of the imaging device 200. The control unit 213 controls the parameters of the image sensor 101. In another embodiment, the shutter speed of the electronic shutter of the image sensor 101 is controlled.

  The configuration of an example of the conversion processing unit 201, the preprocessing circuit 202, and the camera signal processing circuit 203 is shown in FIG. Imaging data from the image sensor 101 (including CDS and A / D converter) is supplied to the conversion processing unit 201. The conversion processing unit 201 includes switches SW 1 and SW 2, a thinning-out unit 221, and a size adjustment unit 222. The thinning unit 221 performs display thinning. The size of the image displayed by the size adjustment unit 222 is changed to an appropriate size. The thinning unit 221 or the thinning unit 221 and the size adjustment unit 222 lowers the screen rate of the RAW data during high-speed imaging to the normal rate. The switches SW1 and SW2 are switched between recording and reproduction. A terminal selected at the time of recording is indicated by r, and a terminal selected at the time of reproduction is indicated by p.

  The output image signal of the size adjustment unit 222 of the conversion processing unit 201 is supplied to the shading correction circuit 231 of the preprocessing circuit 202. The shading correction circuit 231 corrects the brightness around the screen from becoming dark. An output signal of the shading correction circuit 231 is supplied to the camera signal processing circuit 203.

  Similarly to the above-described embodiment, the camera signal processing circuit 203 includes, in order from the input side, a gain correction circuit 240, a synchronization circuit 241, a white balance correction unit 242, an aperture correction unit 243, a gamma correction unit 244, and a YC generation. The unit 245 is provided. However, the configuration of the camera signal processing circuit 203 is not limited to that shown in FIG. 4. For example, the order of these components can be changed, or some components can be omitted. In addition, the camera signal processing circuit 203 is not switched in a time-sharing manner according to an image as in the embodiment.

  The functions of the gain correction circuit 240, the synchronization circuit 241, the white balance correction unit 242, the aperture correction unit 243, the gamma correction unit 244, and the YC generation unit 245 are the same as those in the above-described embodiment. That is, the synchronization circuit 241 interpolates missing pixels for each color component, and the output signal of the synchronization circuit 241 is supplied to the white balance correction unit 242. The white balance correction unit 242 corrects an unbalance between colors due to a difference in the color temperature environment of the subject and a difference in sensitivity due to a color filter on the sensor. The aperture correction unit 243 is contour correction that extracts and emphasizes a portion with a large signal change. The gamma correction unit 244 corrects input / output characteristics so that gradation is correctly reproduced when an imaging signal is output to the display unit 112. The YC generation unit 245 generates a luminance signal (Y) and a color difference signal (C). The luminance signal is generated by synthesizing the gamma-corrected RGB signal at a predetermined synthesis ratio. The color difference signal is generated by combining the gamma-corrected RGB signal at a predetermined combining ratio. The generated luminance signal and color difference signal are supplied to the display unit 112 via the display processing circuit 208.

  In the imaging apparatus of FIG. 5, in the high-speed imaging mode, as shown in FIG. 6, the control unit 213 controls the switches SW1 and SW2 of the conversion processing unit 201 to select the terminal r side. FIG. 7 shows a signal flow in the high-speed imaging mode. As shown in FIG. 7, RAW data of 240 fps from the image sensor 101 is supplied to the recording device 111 via the recording device control circuit 210 via the terminal r of the switch SW1. When a recording request for an image captured at high speed is received from the control unit 213 according to a user operation, RAW data is recorded in the recording device 111. In the RAW data, a shutter speed is set for each of N (= 4) images that are temporally continuous.

  Also, the thinning-out unit 221 of the conversion processing unit 201 thins out the quarter so that the RAW data becomes 60 fps. The RAW data subjected to the thinning and size conversion is supplied to the camera signal processing circuit 203 via the preprocessing circuit 202. A signal subjected to camera signal processing by the camera signal processing circuit 203 is supplied to the display unit 112 via the display processing circuit 208, and an image being captured is displayed on the display unit 112. The displayed image is taken at one of four types of shutter speeds.

  In the imaging apparatus of FIG. 5, at the time of reproduction, the control unit 213 controls the switches SW1 and SW2 of the conversion processing unit 201 to select the terminal p side. FIG. 8 shows the signal flow during reproduction. As shown in FIG. 8, RAW data is read from the recording device 111 at a low screen rate (for example, 60 fps) in accordance with the display capability of the display unit 112 under the control of the recording device control circuit 210. The read RAW data is supplied from the recording device control circuit 210 to the preprocessing circuit 202 via the switches SW1 and SW2 and the size adjustment unit 222 of the conversion processing unit 201.

  An output signal of the preprocessing circuit 202 is supplied to the display unit 112 via the camera signal processing circuit 203 and the display processing circuit 208, and a reproduced image is displayed on the display unit 112. For example, when only the screen rate is changed, the reproduced image is a slow motion reproduced image whose time axis is enlarged four times as compared with the time of recording. It is also possible to perform imaging with four different imaging conditions (exposure conditions, etc.) during imaging, and compare four types of captured images (either still images or moving images) during reproduction. Further, it is also possible to obtain a frame-by-frame playback image by performing thinning when reading the recording device 111.

Note that RAW data recorded on the recording device 111 is, for example, JPEG (Joint
Compressed and encoded processing is performed by an encoding method such as Photographic Experts Group), the compressed encoded data is written to the recording device 111 under the control of the recording device control circuit 210, and the compressed encoded data read from the recording device 111 is expanded. You may make it process.

  In the other embodiment of the present invention described above, normal low screen rate (60 fps) RAW data is supplied to the camera signal processing circuit 203. It cannot be changed in sync with the data. However, since the camera signal processing circuit 203 only needs to operate at a low screen rate, it is possible to avoid an increase in circuit size and an increase in power consumption in order to enable high-speed processing. However, it is possible to change the processing parameters for the RAW data with a high screen rate recorded in the recording device 111 by a device other than the imaging device 200 or a computer.

  The control unit 213 controls the setting of the shutter speed of the image sensor 101 in synchronization with the high-screen-rate RAW data. Since the image sensor 101 is configured as an electronic shutter, a shutter speed can be set for each frame of RAW data having a screen rate of 240 fps (however, the upper limit of the settable shutter speed is 1/240. Second). For example, a different shutter speed is set for each image (any one of A to D) of the RAW data 122 shown in FIG.

  As an example, a shutter speed of 1/240 (second) is set for the image A in the RAW data, and a shutter speed of 1/400 (second) is set for the image B. On the other hand, a shutter speed of 1/600 (second) is set, and for image D, a shutter speed of 1/1000 (second) is set. The set shutter speed is recorded in the recording device 111 in association with the RAW data.

  When writing RAW data with a high screen rate to the recording device 111 or reading RAW data with a low screen rate from the recording device 111, four moving image data are formed from the RAW data by address control. The shutter speed set for each moving image data is read as additional information related to the moving image data. The control unit 213 automatically corrects the gain of the gain correction circuit 240 in the camera signal processing circuit 203 based on the shutter speed of the RAW data read from the recording device 111 and supplied to the camera signal processing circuit 203. That is, gain correction is performed so that an image with appropriate brightness can be obtained by correcting a shortage of incident light amount due to an increase in shutter speed.

  The user can select one RAW data among the four RAW data read from the recording device 111 by operating an input operation unit (not shown) associated with the control unit 213 during reproduction. A reproduced image can be displayed on the unit 112. The user can know an appropriate shutter speed by displaying images with four different shutter speed settings. As described above, image data for which the shutter speed is determined to be appropriate may be recorded on a recording device such as an optical disk or output to the outside via a communication interface, as in the above-described embodiment. Further, the RAW data may be output to the outside via a communication interface instead of being recorded on the recording device.

  The present invention is not limited to the above-described embodiment, and various modifications based on the technical idea of the present invention are possible. For example, the present invention can be applied not only to a camcorder and a digital still camera, but also to an apparatus having an imaging function such as a mobile phone and a PDA (Personal Digital Assistants). Further, the present invention can also be applied to a processing device and a recording device for an imaging signal by a small camera for videophone or game software connected to a personal computer or the like.

It is a block diagram of the imaging device by one embodiment of this invention. It is a basic diagram used for description of an example of the image sensor which can apply this invention. It is a block diagram which shows the more detailed structure of a part of one embodiment of this invention. It is a timing chart which shows the signal processing of one embodiment of this invention. It is a block diagram of the imaging device by other embodiment of this invention. It is a block diagram which shows the more detailed structure of a part of other embodiment of this invention. It is a data flow figure in the high-speed imaging mode of other embodiment of this invention. It is a data flow figure at the time of the image reproduction of other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100, 200 ... Imaging device 101 ... Image sensor 102, 202 ... Pre-processing circuit 103, 203 ... Camera signal processing circuit 104, 201 ... Conversion processing part 105, 109 ... Compression / decompression Circuit 106 ... Memory control circuit 107 ... Memory 108, 208 ... Display processing circuit 111 ... Recording device 112 ... Display unit 113, 213 ... Control unit

Claims (14)

Camera signal processing means for performing image processing on imaging data of a first screen rate obtained from a solid-state imaging device;
First compression encoding means for compressing the image data of the first screen rate processed by the camera signal processing means;
Temporary storage means for temporarily storing the image data encoded by the first compression encoding means;
Decoding means for reading image data captured during a predetermined time from the temporary storage means at a second screen rate lower than the first screen rate, and decompressing the read image data;
Second compression encoding means for compressing the image data expanded by the decoding means;
Storage means for storing the image data encoded by the second compression encoding means;
An image pickup apparatus comprising: control means for controlling the camera signal processing means so that the processing parameters are switched in a time division manner for each of one or a plurality of images.   The imaging apparatus according to claim 1, wherein at least one of the temporary storage unit and the storage unit can be attached and detached.   The imaging apparatus according to claim 1, further comprising display means for displaying image data photographed during the predetermined time read from the temporary storage means at the second screen rate.   When writing image data captured during the predetermined time to the temporary storage means or reading the image data from the temporary storage means, the temporary storage means is controlled to control a plurality of moving images from the image data. The imaging apparatus according to claim 1, wherein data is formed. The first screen rate is N times the second screen rate (N is an integer of 2 or more);
The processing parameter of the camera signal processing means is controlled to be settable for each of N consecutive images of the imaging data of the first screen rate in time, and N moving image data are formed. Item 5. The imaging device according to Item 4.   The imaging apparatus according to claim 4, wherein the moving image data selected from the plurality of moving image data is stored in the storage unit. A camera signal processing step of performing image processing on imaging data of a first screen rate obtained from a solid-state imaging device;
A first compression encoding step for compressing the image data of the first screen rate processed by the camera signal processing step;
A temporary storage step for temporarily storing the image data encoded by the first compression encoding step;
A decoding step of reading image data captured during a predetermined time from the temporary storage step at a second screen rate lower than the first screen rate, and decompressing the read image data;
A second compression encoding step for compressing the image data expanded by the decoding step;
A storage step for storing the image data encoded by the second compression encoding step;
An imaging method comprising: a control step for controlling the camera signal processing step so that the processing parameter is switched in a time division manner for each of one or a plurality of images. Recording medium control means for recording imaging data of a first screen rate obtained from a solid-state imaging device on a recording medium, and reading the imaging data from the recording medium at a second screen rate lower than the first screen rate;
Conversion means for converting imaging data of the first screen rate from the solid-state imaging device into imaging data of the second screen rate;
Camera signal processing means for performing camera signal processing on the imaging data of the second screen rate from the conversion means or the recording medium control means;
Display processing means for generating an image signal for display from the image signal from the camera signal processing means;
An image pickup apparatus comprising: control means for controlling the solid-state image pickup device so that the parameters of the solid-state image pickup device are switched in a time division manner for each of one or a plurality of images.   The imaging apparatus according to claim 8, wherein the parameter of the solid-state imaging device is a shutter speed of an electronic shutter.   The imaging apparatus according to claim 8, wherein the recording medium is attachable and detachable.   A plurality of moving image imaging data is formed from the imaging data by controlling the recording medium control means when writing the imaging data to the recording medium or reading the imaging data from the recording medium. 8. The imaging device according to 8. The first screen rate is N times the second screen rate (N is an integer of 2 or more);
12. The parameter of the solid-state image sensor is controlled to be settable for each of N time-sequential images of imaging data at the first screen rate, and N moving image imaging data are formed. Imaging device.   The imaging apparatus according to claim 8, further comprising compression / expansion means for compressing and encoding imaging data recorded on the recording medium and decompressing and decoding imaging data read from the recording medium. A recording medium control step of recording imaging data of a first screen rate obtained from a solid-state imaging device on a recording medium, and reading the imaging data from the recording medium at a second screen rate lower than the first screen rate;
Converting the imaging data of the first screen rate from the solid-state imaging device into imaging data of the second screen rate;
A camera signal processing step for performing camera signal processing on the imaging data of the second screen rate from the conversion step or the recording medium control step;
A display processing step of generating an image signal for display from the image signal from the camera signal processing step;
An imaging method comprising: a control step of controlling the solid-state image sensor so that the parameters of the solid-state image sensor are switched in a time division manner for each of one or a plurality of images.

Images