JP2006197548A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus in which the variation of the amount of compressed data is reduced, the processing time is reduced, and the capacity requirement of a buffer memory is reduced. <P>SOLUTION: The imaging apparatus includes: an imaging part 1 for picking up a subject image to generate image data and reading out the generated one frame image data that are divided into a plurality of fields; a storage part 3 for temporarily storing the image data obtained by performing predetermined processing on the image data that have been read out; and an image processing part 2 for generating record image data and auxiliary image data for a use other than recording based on the image data that have been stored in the storage part. The record image data are generated using image data of all of the fields of the one frame of the image data, and the auxiliary image data are generated using image data of a subset of the fields of the one frame of the image data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging device such as a digital still camera that records a still image by compression encoding.

  A digital still camera compresses and encodes captured image information using a method such as JPEG, and records the compressed and encoded data (hereinafter referred to as compressed data) in a recording device such as a nonvolatile semiconductor memory such as a flash memory. To do. Since the capacity of the recording device is finite, if the amount of compressed image data varies, the number of recorded images varies, which is not preferable. Therefore, in order to align the data amount per sheet, provisional compression encoding is performed before compression encoding, and an appropriate compression rate is calculated based on the code amount of the generated compressed data. Hereinafter, this is referred to as code amount estimation.

  Hereinafter, processing in a conventional digital still camera will be described with reference to the drawings.

  FIG. 14 is a block diagram of a conventional digital still camera. In FIG. 14, reference numeral 101 denotes an image sensor, which is driven based on a signal from a TG (timing generator) 102. As the pixel array of the image sensor 101, for example, an RGB Bayer array as shown in FIG. 15 is used. The image sensor 101 is an interlaced readout CCD image sensor that reads out an image signal of one frame in three fields. One frame is composed of three fields A to C. In the A field, the charges for every three lines are transferred to the vertical CCD and read sequentially, like lines 1, 4,. Similarly, in the B field, the charges on the lines 2, 5,..., And in the C field, the charges on the lines 3, 6,.

  A RAW data generation unit 103 in FIG. 14 converts a signal read from the image sensor 101 into a digital signal, and generates RAW data. In this specification, RAW data refers to image data that has been read from an image sensor and has undergone necessary processing, and has not been converted to YC data. The RAW data of the A, B, and C fields is once stored in the buffer memory 104, and YC data is generated by the YC processing unit 105. YC data is a signal obtained by superimposing a luminance signal (Y signal) and a color signal (C signal). The YC data processing needs to be performed in the order of the original pixel arrangement shown in FIG. For this reason, the RAW data of each field of A, B, and C is stored every three lines in one area of the buffer memory 104 and is read out line by line, or the A, B, and C fields are stored in separate areas. Thus, it is necessary to read out the data of each field alternately for each line. The YC data generated by the YC processing unit 105 is stored in the YC data area of the buffer memory 104.

  The YC data recorded in the buffer memory 104 is compression encoded by the compression encoding unit 106 using a method such as JPEG. The compressed data is stored in the compressed data area of the buffer memory 104 at the time of main compression, but is not stored in the buffer memory 104 at the time of code amount estimation, and only the information on the generated code amount is compressed from the compression encoding unit 106. It is supplied to the rate calculation unit 107. The compression rate calculation unit 107 calculates an appropriate compression rate based on the code amount obtained by code amount estimation. At the time of the main compression, the compression rate is set in the compression encoding unit 106. At the time of code amount estimation, the compression encoding unit 106 performs compression encoding at a temporary compression rate.

  The compressed data stored in the buffer memory 104 is recorded in a recording unit 108 formed of a recording device such as a semiconductor memory. The control unit 109 controls the TG 101, the RAW data generation unit 103, the YC processing unit 105, the compression encoding unit 106, the compression rate calculation unit 107, and the recording unit 108 to perform these series of processes.

  Next, an operation until the digital still camera configured as described above records image data for one screen will be described. FIG. 16 is a flowchart showing the operation of the control unit 109. First, an appropriate exposure time is provided according to the shooting situation, and the image sensor 101 is exposed (step S201). Next, the TG 102 is controlled to read out the electric charges accumulated in the image sensor 101 in the order of the A, B, and C fields, and the signal read from the image sensor 101 is converted into RAW data by the RAW data generation unit 103, and the buffer It is stored in the memory 104 (step S202).

  Since this digital still camera uses a CCD image sensor that reads out an image signal of one frame in three fields, when storage of RAW data in the C field is started in the buffer memory 104, three fields are sequentially displayed from the top of the screen. RAW data is ready and YC data processing can be started. Therefore, when storage of RAW data in the C field is started (step S203), the YC processing unit 105 starts conversion from RAW data to YC data (step S204). Also, YC data sequentially generated from the top of the image is compression-encoded by the compression encoding unit 106, and code amount estimation is started (step S205). At this stage, since compression encoding is used for code amount estimation, the compression encoding unit 106 performs compression encoding using a preliminarily stored temporary compression rate, and does not store compressed data in the buffer memory 104.

  When the generation of YC data for one screen is completed and the code amount estimation is also completed (step S206), the compression rate calculation unit 107 calculates an appropriate compression rate from the generated code amount (step S207). The compression encoding unit 106 starts main compression (step s208). When the main compression is finished (step S209), the compressed data stored in the buffer memory 104 is recorded in the recording unit 108 (step S210), and the control of the control unit 109 for one screen is finished.

  FIG. 17 is a diagram showing the usage amount of the buffer memory 104 in the above configuration. Reference numeral 110 denotes the amount used by RAW data. As the RAW data of the A, B, and C fields are stored in the buffer memory 104, the usage amount 110 increases. However, since the YC data processing starts in the C field, unnecessary RAW data is discarded from the buffer memory 104 as YC data is generated. Since the amount of RAW data that becomes unnecessary by the YC data processing is larger than the amount of RAW data stored in the C field in the buffer memory 104, the usage amount 110 of the buffer memory 104 slightly decreases. Thereafter, when the storage of the RAW data in the C field is completed, the RAW data that has become unnecessary by the YC data processing is discarded, and the usage amount 110 decreases. As the generation of YC data ends, the usage amount 110 based on RAW data becomes zero.

  Reference numeral 111 denotes the usage amount of the buffer memory 104 based on YC data. The usage amount 111 increases as the Y-field data is stored in the buffer memory 104. Reference numeral 112 denotes a usage amount based on the compressed data. The code amount estimation is started almost simultaneously with the YC data generation, but the compressed data generated thereby is not stored in the buffer memory 104. Therefore, when the code amount is estimated, the use amount 112 by the compressed data is zero. . After the YC data generation and the code amount estimation are completed, the compression encoding unit 106 starts the main compression, and the usage amount 112 based on the compressed data increases. On the other hand, unnecessary YC data is discarded from the buffer memory 104 and the usage amount 111 decreases. When the generation of compressed data ends, the usage amount 111 becomes zero. The compressed data stored in the buffer memory 104 is recorded in the recording unit 108, and the compressed data that is no longer needed is discarded from the buffer memory 104, and the usage amount 112 is drastically reduced, and recording to the recording unit 108 is completed. The usage amount 112 becomes zero.

  As described above, in the conventional digital still camera, in order to prevent variation in the code amount per frame, the code amount is estimated together with the YC data generation, and the main compression is started after the code amount estimation is completed. . That is, compression encoding is performed twice, and the processing time becomes long. Further, since it is necessary to retain the YC data with the largest use amount of the buffer memory at least until the main compression is started, it is necessary to secure the capacity of the buffer memory correspondingly. Also, in order to realize a continuous shooting function, etc., whether to increase the memory capacity with an increase in cost in order to secure further capacity, or to increase the shooting interval to the extent that the amount of YC data used decreases. Had to choose one.

  Therefore, for example, as described in Patent Document 1 and Patent Document 2, a method has been proposed that enables a reduction in processing time and a reduction in the capacity of the buffer memory while performing code amount estimation.

  In the method described in Patent Document 1, input image data is divided into a plurality of small images, and YC data processing and code amount estimation are performed in advance on the data thinned out in units of small images. Shortening and buffer memory capacity reduction are realized. For example, when the data is thinned in half, the time for code amount estimation can be reduced to about half, and the start of the main compression can be advanced correspondingly. Further, since compression encoding can be started when half of the YC data processing is completed, the amount of YC data held in the buffer memory may be halved, and the capacity can be reduced.

In the method described in Patent Document 2, RAW data recorded in a buffer memory is compression-encoded with a single color, and the compression rate of main compression is determined based on the code amount. As a result, storage of the RAW data in the buffer memory and code amount estimation can be performed in parallel, so that the processing time can be shortened.
Japanese Patent Laid-Open No. 2003-304491 JP 2004-088294 A

  However, the above prior art has the following problems.

  In the method described in Patent Document 1, since data is thinned out using a small image group having a predetermined number of pixels and lines, an error in code amount estimation becomes large when there is a large difference in the amount of image data before and after thinning. Probability is high.

  Further, in the method described in Patent Document 2, since the code amount estimation is performed in a single color, there is a high possibility that the error in the code amount estimation becomes large when the amount of color information is large.

  An object of the present invention is to solve the above-described problems and to provide an imaging apparatus with small variations in the amount of compressed data, a short processing time, and a small buffer memory capacity.

  An imaging apparatus according to the present invention captures a subject to generate image data, reads the generated image data of one frame into a plurality of fields, and performs predetermined processing on the read image data. A storage unit for temporarily storing the applied image data, and image processing for generating auxiliary image data for use different from recording image data and recording based on the image data stored in the storage unit A part. The recording image data is generated using image data of all fields in one frame of image data, and the auxiliary image data is generated using image data of some fields of image data of one frame. Is done.

  According to the image pickup apparatus of the present invention, the code amount estimation can be completed before the image data for one frame is stored in the storage unit, and the recording image data needs to be held in the storage unit. almost none. Accordingly, it is possible to provide an imaging apparatus in which the variation in the amount of compressed data is small, the processing time is short, and the required capacity of the buffer memory is small.

  In the imaging apparatus according to the aspect of the invention, the image processing unit may generate the recording image data after generating the auxiliary image data.

  Further, the image processing unit may be configured to perform compression processing when generating the recording image data, and to adjust compression parameters in advance using the auxiliary image data before the compression processing. .

  The image processing unit generates the auxiliary image data by compressing the image data of the partial field, and uses the data size of the auxiliary image data to perform the compression processing of the recording image data. It can be set as the structure which adjusts the compression parameter used in the case.

  The said structure WHEREIN: The parameter memory | storage part which memorize | stores the said adjusted compression parameter can further be provided.

  The thumbnail image data may be generated based on the auxiliary image data.

  Further, a display image based on the auxiliary image data can be displayed on an image monitor.

  The imaging unit may include an AD converter that converts the generated image data from an analog format to a digital format.

  The image processing unit includes a preprocessing unit that performs preprocessing on the image data generated by the imaging unit, and a YC processing unit that converts the processed image data into YC data including a luminance signal and a color difference signal. And a compression unit that compresses the converted YC data.

  Embodiments of the present invention will be specifically described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a digital still camera according to Embodiment 1 of the present invention. Note that solid line arrows in the figure indicate transmission of image signals, and broken line arrows indicate transmission of control signals.

  This digital still camera includes a CCD image sensor 1 that constitutes an imaging unit, an image processing unit 2 that processes electrical signals output from the CCD image sensor 1, and a buffer memory that temporarily stores signals processed by the image processing unit 2. 3 and a controller 4 for controlling the overall operation. The CCD image sensor 1 operates based on timing pulses supplied from a TG (timing generator) 5.

  The image processing unit 2 includes a preprocessing unit 6, a YC processing unit 7, an enlargement / reduction processing unit 8, and a compression encoding unit 9. The output signal of the compression encoding unit 9 is supplied to the code amount counter 10, and the output signal of the code amount counter 10 is supplied to the compression parameter calculation unit 11. The output signal of the compression parameter calculation unit 11 is supplied to the controller 4 and used for control. A memory slot 12 is connected to the buffer memory 3, and a memory card 13 is inserted, so that the compressed data stored in the buffer memory 3 can be recorded on the memory card 13.

  The CCD image sensor 1 decomposes an optical image incident through the optical system 14 into RGB three primary colors, outputs a pixel signal, and supplies the pixel signal to the preprocessing unit 6. The preprocessing unit 6 converts the pixel signal into a digital signal, performs gain adjustment by pedestal processing, white balance (AWB) processing, and the like to create RAW data and stores it in the buffer memory 3.

  FIG. 2 is a pixel array example of the CCD image sensor 1 in the present embodiment. In FIG. 2, the same parts as those in the conventional example of FIG. In a CCD image sensor having a pixel array with a two-line period as shown in (a), signals read out every three lines such as lines 1, 4,... In the A field are as shown in (b). Become. Since each pixel is read out from a pixel array with a two-line period, it has R, G, and B color components only with the A field signal, and it is possible to estimate the code amount including not only the luminance but also the color. .

  The YC processing unit 7 in FIG. 1 acquires RAW data from the buffer memory 3 under the control of the controller 4 and converts the RAW data into YC data.

  The enlargement / reduction processing unit 8 is provided to generate a thumbnail image. In other words, when playing back the recorded compressed data on a liquid crystal display mounted on the main unit, thumbnail images are displayed on the display so that it is convenient to select a playback image from a large number of recorded images. It has a function. Therefore, a thumbnail image is generated in advance by the enlargement / reduction processing unit 8 during shooting and recording.

  The compression encoding unit 9 compresses and encodes the YC data recorded in the buffer memory 3 by a method such as JPEG. The compressed data is stored in the compressed data area of the buffer memory 3 during the main compression. On the other hand, in the code amount estimation process, the compressed data is not stored in the buffer memory 3, and only the generated code amount information is supplied from the compression encoding unit 9 to the code amount counter 10. The code amount counter 10 measures and stores the code amount for a predetermined number of fields. The compression parameter calculation unit 11 calculates an appropriate compression parameter based on the code amount measured by the code amount counter 10. During the main compression, the compression parameter is set in the compression encoding unit 9. At the time of code amount estimation, the compression encoding unit 9 performs compression encoding based on temporary compression parameters.

  In this way, the code amount is once stored in the code amount counter 10 in this embodiment because the code amount is estimated using only the A field, and the main compression is not started immediately after the code amount estimation is completed. This is because the code amount needs to be held until the main compression is started.

  Next, the operation of the digital still camera according to the present embodiment will be described with reference to the flowchart shown in FIG. 3 and the timing chart shown in FIG. In the timing chart shown in FIG. 4, the horizontal axis (m) indicates time, and (a) to (g) indicate the timing of each operation. Also, (h), (j), and (k) show the used capacity of the buffer memory 3.

  First, at time T1, the CCD image sensor 1 is exposed by providing an appropriate exposure time according to the shooting situation (step S1). Next, from time T 2, the charges accumulated in the CCD image sensor 1 are read in the order of the A, B, and C fields, and the read signal is converted into RAW data by the preprocessing unit 6. Store (step S2). In this embodiment, after starting the storage of the RAW data, when the RAW data having the number of lines necessary for the YC data processing is stored in the buffer memory 3, the estimation YC data processing is started (step S3). In FIG. 4, (a) storage of RAW data, (b) code amount estimation, and (c) generation of YC data are illustrated to start at the same time T2. Since it is difficult to illustrate a slight time difference, this is only shown at the same time for convenience, and the operation is actually started sequentially as described above.

  The estimation YC data processing is performed on the RAW data of the A field shown in FIG. 2, and when YC data of the number of lines necessary for compression encoding is stored in the buffer memory (time T3), the code amount estimation is performed. Generation of compressed data is started (step S4). When the storage of the RAW data in the A field ends, the estimation YC data processing ends accordingly (time T4). Further, the generation of the code amount estimation compressed data is completed (time T5), and the code amount estimation is completed (step S5). The code amount generated as a result is stored in the code amount counter 10.

  Subsequently, the storage of the RAW data in the B field is terminated, and the storage of the RAW data in the C field is started at time T6 (step S6). Accordingly, YC data processing for main compression is started (step S7). At time T7, the generation of thumbnail images is started (step S8), the code amount stored in the code amount counter 10 is read to the compression parameter calculation unit 11, the compression parameter is calculated (step S9), and compression encoding is performed. A compression parameter is set for the unit 9. When YC data having the number of lines necessary for compression encoding is stored in the buffer memory 3, the main compression is started (step S10). The compression encoding unit 9 stores the compressed data in the buffer memory 3, and when the main compression is completed at time T9 (step S11), a recording file is generated (step S12). The recording file is recorded on the memory card 13 (step S13), and when the recording ends at time T10, the processing for one screen ends.

  As described above, in the present embodiment, the main compression can be performed in parallel with the YC data processing. Therefore, compared with the conventional example in which the main compression cannot be started until the YC data processing and the code amount estimation are completed, one screen is used. Can be completed quickly.

  In this specification, YC data generated using image data of all fields constituting one frame, or data recorded on a recording medium such as a memory card by compressing and encoding the YC data is used in this specification. This is called data. On the other hand, YC data generated from image data of some fields (in this embodiment, one field) for estimation YC data processing is referred to as auxiliary image data. The auxiliary image data may be compressed data obtained by performing compression encoding processing on YC data. The auxiliary image data can also be used for purposes other than code amount estimation as image data for a different purpose from that for recording.

  Next, a change in the usage amount of the buffer memory 3 will be described with reference to (h), (j), and (k) of FIG. (H) shows the amount of use by RAW data. As time RAW data in the A, B, and C fields are stored in the buffer memory 3 from time T2, the usage amount (h) increases. However, since the YC data processing starts in the C field, unnecessary RAW data is discarded from the buffer memory 3 as YC data is generated. Since the amount of RAW data that becomes unnecessary by the YC data processing is larger than the amount of RAW data stored in the C field in the buffer memory 3, the amount of use of the buffer memory 3 slightly decreases. After that, when the storage of the RAW data in the C field is completed, the RAW data that is no longer needed by the YC data processing is discarded, and the usage amount (h) decreases rapidly. As the generation of YC data ends, the amount of raw data used (h) becomes zero.

  (J) shows the usage amount of the buffer memory 3 by YC data. The amount used (j) is small in this embodiment. The reason is as follows. First, the YC data at the time of code amount estimation is generated from RAW data of only the A field and is necessary only for code amount estimation. YC data is sequentially generated from the upper part of the screen. When YC data having the number of lines necessary for compression encoding, for example, JPEG, is generated with the number of macroblock lines, compression encoding is started. When the macroblock compression encoding is completed, the YC data becomes unnecessary and is discarded from the buffer memory 3. Therefore, the YC data that needs to be held in the buffer memory 3 is about the number of macroblock lines. Next, regarding the YC data at the time of the main compression, in the conventional example, since the main compression is performed after the YC data is generated, it is necessary to hold the YC data in the buffer memory 3 until the main compression is started. On the other hand, in the present embodiment, since compression encoding is started together with YC data generation, if only YC data of about several lines is held in the buffer memory 3 for the same reason as the code amount estimation. Good.

  (K) is the amount used by the compressed data. Similar to the conventional example, the compressed data at the time of code amount estimation is not stored in the buffer memory 3, but only the main compression data is stored, and the compressed data that is recorded on the memory card 13 and becomes unnecessary is stored in the buffer memory 3. Discarded.

  As described above, in the present embodiment, there is almost no need to hold YC data in the buffer memory 3, so that the capacity of the buffer memory 3 can be reduced as compared with the conventional example.

  According to the present embodiment, since the code amount is estimated based on the luminance and color of the entire screen, there is provided a digital still camera in which the amount of compressed data is small, the processing time is short, and the buffer memory capacity is small. I can do it.

(Embodiment 2)
FIG. 5 is a block diagram of a digital still camera according to Embodiment 2 of the present invention. In FIG. 5, the same components as those in FIG. While the first embodiment stores the code amount obtained by the code amount estimation, in this embodiment, after the code amount estimation is completed, the compression parameter calculation unit 11 selects an appropriate compression parameter from the obtained code amount. The calculated compression parameter is stored in the compression parameter storage unit 14. At the time of the main compression, a compression parameter is set for the compression encoding unit 9 based on the compression parameter stored in the compression parameter storage unit 14.

  FIG. 6 is a flowchart showing the operation of the digital still camera in the present embodiment. In FIG. 6, the same parts as those in FIG. In this embodiment, after the code amount estimation is completed (step S5), the compression parameter is calculated, and the calculated compression parameter is stored (step S14).

  The present embodiment is essentially equivalent to the first embodiment, and the same effect can be obtained.

(Embodiment 3)
A digital still camera according to Embodiment 3 of the present invention has a configuration substantially similar to that of Embodiment 1 shown in FIG. The present embodiment is different from the first embodiment in the operation relating to the generation of thumbnail image data. That is, in the present embodiment, as in the first embodiment, YC data is generated using only the A field for code amount estimation, and thumbnail image data is generated by reducing the YC data.

  The operation of the digital still camera in this embodiment will be described with reference to a flowchart shown in FIG. 7 and a timing chart shown in FIG. 7 and 8, the same parts as those in FIGS. 3 and 4 are denoted by the same reference numerals, and repeated description is omitted.

  In the present embodiment, unlike the first embodiment of FIG. 3, the reduction process is started (step S15) almost simultaneously with the estimation YC data process start (step S3) (time T3). Then, almost simultaneously with the completion of the A field, the estimation YC data processing, and the code amount estimation, the reduction processing is also completed (time T5), and the reduced thumbnail YC data is recorded in the memory card 13. . Thus, instead of directly recording the YC data for the thumbnail, it may be recorded after being compressed and encoded.

  As described above, in the present embodiment, since the YC data for thumbnails is generated only from the RAW data of the A field, it does not take time to generate thumbnail images not only when displaying images but also when shooting. . Further, it is not necessary to store YC data of the original image size in the buffer memory for generating thumbnail images. Therefore, it is possible to provide a digital still camera with a short processing time and a small buffer memory capacity.

  In the present embodiment, the reduction process is used to generate the thumbnail image, but the application is not particularly limited. This embodiment can be applied to a case where another image having a size smaller than an image of one frame is desired to be generated without requiring processing time.

(Embodiment 4)
The digital still camera according to the fourth embodiment of the present invention has an improved configuration for displaying an image on a monitor such as an LCD. Digital still cameras have a function of displaying an image captured by an image sensor on an LCD or the like, and many digital still cameras do not have an optical viewfinder. Therefore, a shooting style in which the shutter is pressed while looking at the LCD instead of looking through the viewfinder is generally used, and the display quality of the LCD affects the success or failure of shooting.

  The imaging unit of the digital still camera has a monitor mode and a still mode, and the operation differs depending on the mode. In the monitor mode, input image data for one screen is output every 1 / 60th of a second, but the number of output pixels is small and the resolution is the same as that of a video camera. On the other hand, in the still mode, image data with higher resolution and higher pixels is output. However, since the number of pixels is large, a relatively long time is required for output. Therefore, it takes a long time (display time lag) until image data is read out by imaging and displayed on the LCD. In the monitor mode, the display time lag is small and is about 1/30 second, whereas in the still mode, the display time lag is from 1/4 second to 1/2 second and is very long.

  If the LCD display is delayed, the movement of the subject is delayed. In particular, the continuous shooting mode in which still shooting is performed continuously is intended to shoot a moving subject, so that if the LCD display is delayed, it becomes difficult to operate the camera in accordance with the movement of the subject. As a result, an image that does not correctly capture the subject is recorded.

  The reason why the display time lag becomes longer in the still mode will be described with reference to FIG. FIG. 18 shows an image display operation on the LCD in the still mode. The upper row shows the time change of the image data X in the RGB format. The middle row shows the output period of the display image data (image display data) Y. The lower Z indicates the image display period. The numbers in each cell represent the frame number of the image. If the numbers are the same, it means the same frame. A, B, and C indicate each field.

  The image data X is sequentially output as the fields 1A, 1B, 1C of the first frame, the fields 2A, 2B, 2C,. With the output of the image data X of each C field, the generation of the display image data Y is started, and the image display period Z of the frame starts from the time when the generation of the display image data Y for one frame is completed. Since the display time lag L is from the start of the output of the image data X of the A field to the completion of the generation of the display image data Y, as described above, the display time lag L becomes considerably long. In the present embodiment, by using YC data generated only in the A field for code amount estimation, it is possible to display a monitor image with a short display time lag L.

  FIG. 9 is a block diagram of a digital still camera according to Embodiment 4 of the present invention. 9, the same reference numerals are used for the same components as those in the first embodiment shown in FIG. 1, and duplicate descriptions are omitted.

  This digital still camera is different from the first embodiment shown in FIG. 1 in the display image data generation method for displaying an image by driving the liquid crystal monitor 17 by the liquid crystal driver 16. The configuration for managing the buffer memory 3 via the MMU (memory managing unit) 15 is different from that of the first embodiment.

  The image processing unit 2 operates in two modes, a display image mode and a recording image mode, and the mode is switched by a mode switching signal supplied from the controller 4. In the recording image mode, image data in RGB format is read from the buffer memory 3 via the MMU 15, YC data corresponding to the number of recording pixels is generated, and written back to the YC data area for recording image in the buffer memory 3. In the display image mode, RGB format image data is read from the buffer memory 3 via the MMU 15, YC data corresponding to the number of display pixels is generated, and written back to the display image YC data area of the buffer memory 3.

  The MMU 15 has a function of dynamically allocating memory by on-demand paging and a function of automatically collecting pages that are no longer needed. For example, when the CCD image sensor 1 outputs RGB format image data, one page of memory is automatically allocated to the RGB format image data when the first pixel data is stored. At the same time as writing of the memory for one page is completed, the memory for the next page is newly allocated for image data in the RGB format. Further, when the image processing unit 2 reads RGB format image data, every time the image processing unit 2 finishes reading one page of the RGB format image data, unnecessary pages are automatically collected and empty pages are collected. And will be reused in the next assignment.

  The liquid crystal driver 16 reads the YC data from the display image YC data area of the buffer memory 3 via the MMU 15 and displays it on the liquid crystal monitor 17. The display data area is assigned a fixed page for repeated reading, and the MMU 15 does not automatically collect the page.

  Next, a configuration related to YC data processing of the image processing unit 2 will be described with reference to FIG. FIG. 10 is a block diagram illustrating a partial configuration of the image processing unit 2. A register F group 21 and a register B group 22 are connected to the YC processing unit 7 via a selector 23. Each of the register F group 21 and the register B group 22 instructs the operation of the YC processing unit 7. The contents of the instruction include the start address of the RGB data area and the start address of the YC data area. The outputs of the register F group 21 and the register B group 22 are selected by the selector 23, and only one instruction is given to the YC processing unit 7.

  The YC processing unit 7 reads RGB data, converts data, and stores YC data in accordance with instructions, and outputs a trigger pulse group 25 in synchronization with the reading and storing. This trigger pulse group 25 is a pulse for updating the leading address of RGB data and the leading address of YC data in accordance with the progress of storage and reading. The trigger pulse group 25 is given to only the register group selected by the selector 23 among the register F group 21 and the register B group 22 via the gate circuit 24, and the value of each register is updated. Selection by the selector 23 and the gate circuit 24 is performed by a mode switching signal 26.

  Next, the operation in mode switching will be described. In the recording image mode, the register B group 22 is selected, and the leading address of the RGB data and the leading address of the YC data in the register B group 22 are incremented as the processing proceeds. When the mode switching signal 26 changes to switch to the display image mode at a certain point in time, the register F group 21 is selected instead of the register B group 22, and the register B group 22 does not change. When the mode switching signal 26 changes again to return to the recording image mode, the leading address of the RGB data and the leading address of the YC data in the register B group 22 are kept in a state immediately before the processing of the recorded image is stopped. The YC processing unit 7 can resume the processing of the recorded image from the interrupted location.

  The operation of the digital still camera in this embodiment will be described with reference to a flowchart shown in FIG. 11 and a timing chart shown in FIG. 11 and 12, the same parts as those in FIGS. 3 and 4 are denoted by the same reference numerals, and repeated description is omitted.

  The present embodiment is different from the first embodiment in that the image display (e) is started (step S16) almost simultaneously with the estimation YC data processing (step S5) (time T5 in FIG. 12). To do.

  When the photographing operation is started, as shown in FIG. 12B, the preprocessing unit 6 stores the image data in the RGB format into three fields. At the same time, the preprocessing unit 6 notifies the controller 4 of the stored number of lines as a line number signal, and the controller 4 activates the YC processing unit 7 in the display image mode. As shown in FIG. 12 (d), the YC processing unit 7 reads out RGB format image data from the buffer memory 3 in synchronization with the storage by the preprocessing unit 6 (time T2), and from this, YC matching the number of display pixels. Data is generated and written back to the buffer memory 3. When the generation of YC data for the A field ends (time T4), image display by the frame is started (time T5) as shown in FIG. As a result, the display time lag is shorter than the period for outputting image data in RGB format for two fields, as compared with the conventional example shown in FIG.

  As described above, in the digital still camera according to the present embodiment, YC data is generated twice for each frame. The first generation of YC data is generation of display image YC data, and is generated at the same time as the CCD image sensor 1 outputs RGB data of the first A field, so that the display image can be updated in the shortest time. The second generation of YC data is generation of recorded image YC data, which is generated by referring to RGB data of all fields, so that an image with no resolution degradation can be recorded.

  Next, the operation during continuous shooting will be described with reference to FIG. This figure shows the same time chart as FIG. 12 continuously displayed for two frames. However, the used capacity (h) of the buffer memory shows the used capacity of RAW data, YC data, and compressed data in an overlapping manner.

  In the continuous shooting operation, the image data generation period for recording that ends at time T10 in FIG. 13F and the output of RGB format image data of the A field of the next frame that starts at time T10 in FIG. 13B are output. The period, that is, the generation period of the display image data may overlap. Both periods overlap when the number of pixels in the recorded image is large, and the compressed data generation process does not end before the start of storage of RAW data in the A field of the next frame.

  On the other hand, in the digital still camera according to the present embodiment, the image processing unit 2 has two modes, a display image mode and a recording image mode. By switching the modes alternately, the display image data and the recording image data are displayed. It can be generated in a pseudo parallel manner. At this time, the controller 4 controls the mode switching signal so that the display image is generated with priority. Specifically, when the preprocessing unit 6 outputs 64 lines of image data, the controller 4 switches the mode switching signal to set the YC processing unit 7 to the display image mode, and the YC processing unit 7 processes the image data for 64 lines. Control is performed so that the mode switching signal is returned to the recording image mode side after completion of the operation. By such control, the display image is generated without delay, and the remaining processing time is allocated to generation of the recorded image of the previous frame without waste.

  If the YC processing unit 7 does not have the function of performing time-division parallel processing, the generation of the recording image is temporarily stopped in the middle of the A field of the next frame, and the display image generation is started. The generation of the recorded image may be resumed when the generation of the image is completed.

  The operation of the compression encoding unit 9 in the second frame is slightly irregular because the compression encoding unit 9 does not have the function of time division parallel processing like the YC processing unit 7. First, when the display image data of the second frame is generated, since the compression encoding of the recording image data of the previous frame is not completed, the compression encoding of the display image data of the second frame is started after the completion. To do. Similarly to the first frame, the second frame optimizes the compression coding parameters according to the result of the compression coding of the display image. The compression encoding unit 9 waits for the YC processing unit 7 to start generating the recording image data for the second frame, and starts compression encoding of the recording image data. At that time, the compression encoding of the display image data is performed. Use the parameters optimized by

  As described above, the timing of the processing operation of the compression encoding unit 9 of the second frame may not necessarily coincide with the timing of the YC processing unit 7 generating the recorded image data. It does not cause an uncomfortable appearance like the generation, and the compression encoding of the second display image data only needs to be completed before the start of the compression encoding of the second recording image data. Even if there is no time-division parallel processing function like the YC processing unit 7, there is no problem.

  In this embodiment, the parameters are optimized based on the result of compressing and encoding the display image data. However, when the difference in the number of pixels between the recorded image data and the display image data is large, the prediction error of the JPEG data amount increases. Is also a concern. In that case, a trial image for compression encoding can be generated at the same time as the generation of display image data or before and after the generation of display image data, and the parameters can be optimized based on the result of compression encoding the trial image. . If the trial image is an image having the number of pixels that matches the number of recorded pixels, the prediction error of the JPEG data amount can be reduced as compared with the case of compressing and encoding display image data having a number of pixels significantly different from the recorded image data. I can do it.

  As another method, only the trial image is generated without generating the display image data, and the liquid crystal driver 16 thins out the pixel data of the trial image and displays it on the liquid crystal monitor. When this method is adopted, if the number of pixels in the horizontal and vertical directions of the trial image is set to an integral multiple of the number of pixels in the horizontal and vertical directions of the display image, the thinning operation can be easily performed. .

  In the above embodiment, auxiliary image data (YC data) is generated based on image data of only the A field. However, YC data is generated based on image data of a plurality of fields in one frame. It may be generated. Thereby, the accuracy of code amount estimation can be improved. Further, it is desirable to generate auxiliary image data using image data of a plurality of fields even when data of all colors is not obtained with only one field of image data. In order to sufficiently obtain the effects of the invention, it is desirable to generate auxiliary image data using image data of a partial field that is read first.

  The imaging unit is not limited to a CCD image sensor, and a CMOS image sensor or the like can also be used. The AD converter may be built in the imaging unit or may be externally attached.

  The image processing unit may be configured with a hardware circuit like a DSP, or may be configured with a microcomputer using software. The image processing unit, code amount counter, compression parameter calculation unit, and controller can be configured with one chip.

  The buffer memory may be composed of a single memory or may be composed of a plurality of memories. The buffer memory may be directly controlled by the controller as in the first embodiment or the like, or may be controlled using the MMU as in the fourth embodiment.

  The recording memory may be a removable memory card or a built-in memory.

  The RAW data has been described as RGB data, but may be complementary color image data. The image sensor may output YC data instead of RGB data.

  The memory card may be configured to record uncompressed data instead of compressed data.

  According to the imaging apparatus of the present invention, since the variation in the amount of compressed data is small, the processing time is short, and the required capacity of the buffer memory is small, a digital still camera, a digital video camera having a still image shooting function, and a mobile phone with a camera function Useful for terminals.

1 is a block diagram showing a configuration of a digital still camera according to Embodiment 1 of the present invention. The figure which shows the pixel arrangement | sequence of the CCD image sensor which comprises the digital still camera Flow chart showing the operation of the digital still camera Timing chart for explaining the operation of the digital still camera Block diagram showing a configuration of a digital still camera according to Embodiment 2 of the present invention. Flow chart showing the operation of the digital still camera The flowchart which shows the operation | movement of the digital still camera in Embodiment 3 of this invention. Timing chart for explaining the operation of the digital still camera Block diagram showing a configuration of a digital still camera according to Embodiment 4 of the present invention. Block diagram showing a configuration example of a part of an image processing unit constituting the digital still camera Flow chart showing the operation of the digital still camera Timing chart for explaining the operation of the digital still camera Timing chart for explaining the operation of the digital still camera during continuous shooting Block diagram showing the configuration of a conventional digital still camera The figure which shows the pixel array example of the image pick-up element in the digital still camera Flow chart showing the operation of the digital still camera Diagram showing the amount of buffer memory used in the digital still camera The figure which shows the operation of the picture display of the same digital still camera

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 CCD image sensor 2 Image processing part 3 Buffer memory 4 Controller 5 TG (timing generator)
6 Pre-processing unit 7 YC processing unit 8 Enlargement / reduction processing unit 9 Compression encoding unit 10, 10a Code amount counter 11 Compression parameter calculation unit 12 Memory slot 13 Memory card 14 Optical system 15 MMU (memory managing unit)
16 Liquid crystal driver 17 Liquid crystal monitor 21 Register F group 22 Register B group 23 Selector 24 Gate circuit 25 Trigger pulse group 26 Mode switching signal 101 Image sensor 102 TG (timing generator)
193 Raw data generation unit 104 Buffer memory 105 YC processing unit 106 Compression encoding unit 107 Compression rate calculation unit 108 Recording unit 109 Control unit 110 Usage amount based on RAW data 111 Usage amount based on YC data 112 Usage amount based on compressed data X Image data Y Display image data Z Image display period

Claims (9)

An imaging unit that captures an image of a subject, generates image data, and reads the generated image data of one frame into a plurality of fields;
A storage unit that temporarily stores image data obtained by performing predetermined processing on the read image data;
An image processing unit for generating auxiliary image data for use different from recording image data and recording based on the image data stored in the storage unit;
The recording image data is generated using image data of all fields in one frame of image data, and the auxiliary image data is generated using image data of some fields of image data of one frame. An imaging device characterized in that the imaging device is provided.   The imaging apparatus according to claim 1, wherein the image processing unit generates the recording image data after generating the auxiliary image data.   The imaging apparatus according to claim 2, wherein the image processing unit performs compression processing when generating the recording image data, and adjusts a compression parameter in advance using the auxiliary image data before the compression processing. .   The image processing unit generates the auxiliary image data by compressing the image data of the partial field, and uses the data size of the auxiliary image data to compress the recording image data. The imaging apparatus according to claim 3, wherein a compression parameter to be used is adjusted.   The imaging apparatus according to claim 3, further comprising a parameter storage unit that stores the adjusted compression parameter.   The imaging apparatus according to claim 1, wherein thumbnail image data is generated based on the auxiliary image data.   The imaging apparatus according to claim 1, wherein a display image based on the auxiliary image data is displayed on an image monitor.   The imaging apparatus according to claim 1, wherein the imaging unit includes an AD converter that converts the generated image data from an analog format to a digital format. The image processing unit
A preprocessing unit that performs preprocessing on the image data generated by the imaging unit;
A YC processing unit for converting the processed image data into YC data composed of a luminance signal and a color difference signal;
The imaging apparatus according to claim 1, further comprising: a compression unit that compresses the converted YC data.

Images