JP2011023928A - Imaging apparatus, image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which performs compression coding of image data to be settled in prescribed size. <P>SOLUTION: The imaging apparatus is provided with: an imaging means 1 for generating first image data which is first resolution; a resolution conversion means 10 for expanding or reducing the image data to arbitrary resolution; a coding means 11 for performing the compression coding to the image data to make the image data into coded image data; and a control means 12 for controlling the imaging means 1, the resolution conversion means 10, and the coding means 11, wherein the control means 12 obtains, when the resolution conversion means 10 is controlled, and the first image data are reduced to second resolution lower than the first resolution to obtain the second image data, the second image data by temporarily reducing either of the first image data or the second image data to third resolution which is lower than the second resolution and thereafter expanding the image data to the second resolution, and the control means 12 obtains the coded image data by making the coding means 11 code the second image data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an image processing apparatus that compress and encode image data and output the image data.

  In an imaging device such as a digital camera that captures moving images and still images, generally, the captured image data is compression-encoded by a predetermined compression encoding method such as a JPEG (Joint Photographic Experts Group) method, and encoded image data. Recorded on a recording medium. Further, when recording the encoded image data on the recording medium, thumbnail image data used as an index image of the encoded image data recorded on the recording medium may be generated and recorded on the recording medium together with the encoded image data. (For example, refer to Patent Document 1). The thumbnail image data is a small image obtained by thinning out pixel data of the image data to roughen the resolution, and is normally recorded on the recording medium as encoded thumbnail image data further subjected to compression encoding.

JP 2001-69465 A

  In an imaging device, for example, the resolution of thumbnail image data (the number of pixels in the horizontal and vertical directions of the image data) and the code for convenience of image format standards such as AVCHD (Advanced Video Code High Definition), and mounting of embedded devices such as digital cameras. An upper limit may be set for the size of the thumbnail image data. In particular, when an upper limit value is set for the data size, when the thumbnail image data is compressed and encoded to obtain the encoded thumbnail image data, the data size needs to be kept below a predetermined size. However, in conventional imaging apparatuses, compression encoding is often performed using a hardware encoding unit, and due to insufficient compression performance of the encoding unit, it may not be possible to fit within a predetermined data size. There was a problem that there was. Hereinafter, this problem will be described.

  The compression rate when compressing and encoding image data using the JPEG method, etc. includes the parameters such as the quantization step set in the encoding unit that performs compression encoding, as well as the complexity of the pattern of the image data that is the object of compression encoding. Depends on the size. Therefore, when it is desired to keep the image data below a predetermined size, the encoding parameters are set so that the data size after compression encoding is below the predetermined size even when complex image data is input. Set. However, normally, the range of parameters that can be set by the encoding unit is determined, and even if the parameters are set so that the compression ratio becomes the highest, when image data of a very complicated picture is input, the predetermined size is set. There were cases where it was not possible to fit the following.

  An object of the present invention is to solve the above-described problems and to provide an imaging apparatus capable of compressing and encoding image data into a predetermined size even when an encoding unit having a limit in compression performance is used. .

  The above object is achieved by the following imaging device.

  An imaging unit that generates first image data having a first resolution, a resolution conversion unit that expands or reduces the image data to an arbitrary resolution, and encoded image data obtained by compressing and encoding the image data And a control means for controlling the imaging means, the resolution conversion means, and the encoding means, and the control means controls the resolution conversion means to lower the first image data to be lower than the first resolution. When obtaining the second image data by reducing the image data to the second resolution, either the first image data or the image data obtained by enlarging or reducing the first image data by the resolution conversion means is selected from the second resolution. The image data is once reduced to a lower third resolution, then enlarged to the second resolution and subjected to enlargement / reduction processing to obtain second image data, and the control means encodes the second image data to the encoding means Let me Goka so as to obtain the image data.

  According to the present invention, it is possible to provide an imaging apparatus capable of being stored in a predetermined size when image data is compressed and encoded even when an encoding unit having a limit in compression performance is used.

1 is a block diagram showing a configuration of an imaging apparatus according to a first embodiment. 6 is a flowchart showing an operation when image data is captured and recorded in the imaging apparatus according to the first embodiment. 6 is a flowchart illustrating an operation when generating and recording thumbnail image data in the imaging apparatus according to the first embodiment. 6 is a flowchart showing the operation of the reduction / enlargement process in the imaging apparatus according to the first embodiment.

(Embodiment 1)
1. Configuration of Imaging Device According to First Embodiment FIG. 1 is a block diagram illustrating a configuration of an imaging device according to the first embodiment.

  The imaging apparatus according to the present embodiment uses, as a recording medium, encoded image data obtained by compression-encoding captured image data, and encoded thumbnail image data obtained by compression-encoding thumbnail image data obtained by reducing image data. Record on a memory card. The imaging device records the captured image data as a still image and reduces the image data to a thumbnail image. The size of the encoded thumbnail image data is set to be equal to or smaller than a predetermined size set in advance.

  The imaging unit 1 generates image data by converting an optical signal of an image collected by a lens (not shown) into an electrical signal. A gain control unit (hereinafter referred to as AGC) 5 adjusts the gain of the generated image data. The A / D converter 6 digitizes image data that is an analog signal output from the AGC. The YC generation unit 8 converts the digitized image data into a luminance signal and a color signal. The converted image data is temporarily stored in the buffer memory 2 via the memory controller 7. The imaging unit 1 can use a CCD image sensor, a CMOS image sensor, an NMOS image sensor, or the like as an imaging element.

  The resolution conversion unit 10 performs processing for converting the resolution of the image data stored in the buffer memory 2 and generating an enlarged / reduced image. The resolution converter 10 is also used when generating thumbnail image data by reducing the image data. The generated thumbnail image data is temporarily stored in the buffer memory 2 separately from the original image data. The resolution of the thumbnail image (hereinafter referred to as thumbnail resolution) is assumed to be 400 pixels wide by 300 pixels high. When the resolution of the image data output from the imaging unit 1 (hereinafter referred to as imaging resolution) and the resolution of the image data to be recorded on the memory card 4 (hereinafter referred to as recording resolution) are different, the resolution conversion unit 10 performs the enlargement / reduction processing to convert to the set resolution. In this embodiment, the imaging resolution is 4000 pixels wide × 3000 pixels high, and the recording resolution is 2000 pixels wide × 1500 pixels high. Even when the encoded image data recorded on the memory card is decoded and displayed on the display unit 15, the resolution conversion unit 10 operates the operation unit 13 according to the resolution of the display unit 15 or by the user. The enlargement / reduction processing is performed based on the display size specified in the above. The resolution of the image data output from the resolution conversion unit 10 is designated by the microcomputer 12.

  The compression encoding unit 11 compresses and encodes the image data and thumbnail image data with preset encoding parameters, and records them in the memory card 4 as encoded image data and encoded thumbnail image data, respectively. Different encoding parameters can be set for image data and thumbnail image data.

  As described above, the image data stored in the buffer memory 2 is subjected to resolution conversion processing by the resolution conversion unit 10 as necessary, and then compression encoded by the compression encoding unit 11 to obtain encoded image data. It is stored in the memory card 4. Similarly, thumbnail image data generated by reducing the image data is compression-encoded by the compression encoding unit 11 and stored in the memory card 4 as encoded thumbnail image data.

  The decoding unit 17 decodes the encoded image data and the encoded thumbnail image data recorded on the memory card, and stores them in the buffer memory as image data and thumbnail image data, respectively.

  The display unit 15 displays the image data and thumbnail image data stored in the buffer memory 2. An electronic viewfinder (EVF) and a liquid crystal monitor are examples of the display unit 15.

  The memory card 4 is detachable from a slot (not shown) provided in the imaging device. Note that recording may be performed on a recording medium such as an HDD (Hard Disk Drive) or an optical disk instead of the memory card 4. Moreover, these recording media may be provided in the imaging device in a state where they cannot be removed.

  The microcomputer 12 controls these series of operations in accordance with a user instruction via the operation unit 13. The microcomputer 12 executes various operations by reading and executing the program stored in the storage unit 16. Various parameters and the like are also stored in the storage unit 16, and are read and written as necessary. For example, the recording resolution set by the user operating the operation unit 13 is stored in the storage unit 16. When the imaging device records image data on the memory card 4, the microcomputer 12 sets the recording resolution to the resolution conversion unit 10 and performs enlargement / reduction processing to convert the image data of the imaging resolution to the recording resolution. Let it be done. Similarly, when recording thumbnail image data, the microcomputer 12 causes the resolution conversion unit 10 to set a thumbnail resolution and perform a reduction process. Further, the microcomputer 12 sets various parameters for encoding such as a quantization step in the compression encoding unit 11 based on the setting such as the compression rate set by the user.

2. Operation 2-1. Flow Next, the operation of the imaging apparatus according to the present embodiment will be described.

2-1-1. Operation when Recording Image Data First, as a basic operation of the imaging apparatus according to the present embodiment, an operation when recording captured image data on the memory card 4 will be described with reference to a flowchart of FIG. FIG. 2 is a flowchart showing an operation when image data is captured and recorded in the imaging apparatus according to the present embodiment.

  The microcomputer 12 determines various shooting parameters in accordance with instructions from the user via the operation unit 13 (S101). Examples of the shooting parameters include zoom of an optical system (not shown), focus and aperture settings, ISO sensitivity, shutter speed, white balance, recording resolution, compression degree at the time of compression encoding, and the like. All of these can be set by the user, but the microcomputer 12 may determine some or all of the settings. Next, the microcomputer 12 monitors whether or not the shutter button provided on the operation unit 13 has been pressed (S102).

  When the shutter button is pressed, the imaging unit 1 captures an object based on the control of the microcomputer 12 and generates image data (S103). The generated image data is gain-adjusted by the AGC 5 with a gain corresponding to the ISO sensitivity setting, and then converted into a digital signal by the A / D converter 6. The YC generation unit 8 converts the image data digitized by the A / D conversion unit 6 into a luminance signal and a color signal (S104).

  The microcomputer 12 determines the enlargement / reduction ratio corresponding to the recording resolution (S105). The magnification can be obtained as the ratio of the recording resolution to the imaging resolution. For example, if the imaging resolution and recording resolution are both 4000 × 3000 pixels, the magnification is 1 (hereinafter, the magnification is the magnification relative to the number of vertical and horizontal pixels), and the enlargement / reduction processing is not performed. In the present embodiment, since the imaging resolution is 4000 × 3000 pixels and the recording resolution is 2000 × 1500 pixels, the magnification is ½. The magnification needs to be determined for each of the vertical direction and the horizontal direction. In the following description, the same magnification is used for the sake of convenience, but each resolution may be set to have a different magnification. Enlargement / reduction processing is performed in the resolution conversion unit 10 (S107).

  After the enlargement / reduction processing, the image data is compression-encoded by the encoding unit 11 (S110), and encoded image data is generated. The encoding parameters at this time are set based on user settings. The encoded image data is recorded on the memory card 4 (S111). Finally, thumbnail image data corresponding to the image data captured and recorded is generated and recorded in the memory card 4 as encoded thumbnail image data (S112). The generation of thumbnail image data will be described in detail below.

2-1-2. Operation when Recording Thumbnail Image Data Next, the operation when generating and recording thumbnail image data will be described in detail. FIG. 3 is a flowchart showing an operation when generating and recording thumbnail image data in the imaging apparatus according to the present embodiment. In the present embodiment, it is assumed that thumbnail image data is generated from the image data of recording resolution stored in the buffer memory 2 as target image data. The microcomputer 12 performs a reduction process on the image data stored in the buffer memory 2 and controls each unit as follows to generate thumbnail image data.

  First, a reduction ratio for generating thumbnail image data by reducing image data to be processed is determined (S201). If the resolution of the target image data is the recording resolution, the magnification can be obtained as a ratio of the thumbnail resolution to the recording resolution. In this embodiment, since the recording resolution is 2000 × 1500 pixels and the thumbnail resolution is 400 × 300 pixels, the magnification is 1/5.

  After the magnification is determined, the resolution conversion unit 10 performs a reduction process (S203). The thumbnail image data after the resolution conversion process is compression-encoded by the encoding unit 11 (S205), and encoded thumbnail image data is generated. The encoding parameter at this time is set to a combination that can be specified by the encoding unit 11 and has the highest compression rate.

  Here, the microcomputer 12 determines whether or not the size of the encoded thumbnail image data is within a predetermined size (S206). For example, assuming that the predetermined size is 16 kilobytes (KB), the microcomputer 12 compares the size of the compression-encoded encoded thumbnail image data with 16 KB. If it is equal to or smaller than the predetermined size (“Yes” in S206), the encoded thumbnail image data is recorded in the memory card 4 (S207). If it exceeds the predetermined size (“No” in S206), the image data is once reduced to a resolution smaller than the thumbnail resolution (for example, 1/2), and then the reduced image is enlarged again to the size of the thumbnail image data. Then, reduction / enlargement processing is performed (S208), and then compression encoding processing is performed again (S205).

  Next, the reduction / enlargement process will be described in detail. FIG. 4 is a flowchart showing the operation of the reduction / enlargement process in which the target image data is once reduced to a resolution smaller than the thumbnail resolution and then enlarged again to the thumbnail resolution. In this processing, the image data to be processed is temporarily reduced to image data (hereinafter referred to as relay image data) having a lower resolution (hereinafter referred to as relay resolution) than the thumbnail image data desired to be finally obtained. Then, enlargement processing is performed to obtain thumbnail image data. By reducing to a relay resolution lower than the thumbnail resolution and then enlarging to the thumbnail resolution, the substantial information amount of the obtained thumbnail image data becomes equal to that of the relay resolution. As a result, when the encoding unit 11 performs encoding, the data amount can be further reduced.

  In the following description, it is assumed that the image data to be processed is image data of the recording resolution. However, as the image data to be processed, image data of an imaging resolution that has not been subjected to resolution conversion processing to the recording resolution, or already thumbnail resolution. Reduced thumbnail image data may be selected.

  First, the image data to be processed is reduced to image data having a lower resolution than the thumbnail image data to be finally obtained (hereinafter referred to as relay resolution), and the image data is enlarged from there to the thumbnail image data. The enlargement magnification at that time is determined (S301). If the relay resolution is 200 × 150 pixels, the relay resolution magnification (reduction magnification) with respect to the recording resolution (2000 × 1500 pixels) is 1/10, and the thumbnail resolution (400 × 300 pixels) magnification (enlargement) with respect to the relay resolution. (Magnification) is doubled. That is, the thumbnail image data is obtained by temporarily reducing the image data to be processed to half the resolution of the thumbnail image data and then enlarging it twice. When thumbnail image data is selected as the target image data, the reduction magnification multiplied by the enlargement magnification is 1, and in this embodiment, the reduction magnification is ½ and the enlargement magnification is 2.

  After the magnification is determined, the resolution conversion unit 10 performs a reduction process (S303). After the reduction process, the enlargement process is further performed in the resolution conversion unit 10 (S304).

2-2. Effects As described above, the imaging apparatus according to the present embodiment reduces thumbnail image data with a substantial amount of information compared to thumbnail image data obtained only by normal reduction processing by reduction / enlargement processing. Can be obtained. This is due to the loss of high frequency components of the image data in accordance with the reduction magnification. For this reason, even when the compression performance of the encoding unit 11 is limited, the size of the encoded thumbnail image data obtained when the compression encoding is performed can be further reduced and can be kept below a predetermined size. become. Further, the reduction / enlargement processing can be performed by using the resolution conversion unit 10 used in the normal imaging operation, and the encoding unit 11 does not require a special configuration, and thus a large-scale change from the conventional configuration. This can be realized without adding.

  If the size of the encoded thumbnail image data cannot be reduced to a predetermined size or less by the reduction / enlargement processing (“No” in S206), the relay resolution is set to a lower resolution (S301). The reduction / enlargement process may be performed again (S303, S304). In this case, relay image data of relay resolution can be used as target image data in further reduction / enlargement processing.

  Although the imaging apparatus according to the present embodiment has been described as generating thumbnail image data after compressing and recording image data to be recorded as encoded image data, timing for generating thumbnail image data Is not limited to this, and may be any time as long as image data is stored in the buffer memory 2. For example, any time after image data is stored in the buffer memory 2 by the YC generation unit 8 may be used. As for the target image data for thumbnail image generation, in addition to the image data having the imaging resolution and the recording resolution described in the present embodiment, the image data whose resolution is converted for display on the display unit 15 may be used as the target image data.

  In the present embodiment, the image pickup apparatus that generates and records thumbnail image data when image data is captured and recorded has been described. However, thumbnail image data is generated for input image data. The image processing apparatus may be configured. For example, the decoding unit 17 decodes the encoded image data stored in the memory card 4 or the encoded image data transmitted from the outside, and stores the decoded image data in the buffer memory 2 as the target image data. You may comprise as an image processing apparatus which produces | generates thumbnail image data by the process similar to.

  In this embodiment, image data is reduced to thumbnail resolution as usual (without going through relay resolution) to obtain thumbnail image data, and the size of encoded thumbnail image data after compression encoding does not become a predetermined size or less Although the configuration for further reducing the size by the reduction / enlargement processing has been described, the reduction / enlargement processing may be performed unconditionally from the beginning.

In this embodiment, the image data to be processed is once reduced to half the resolution of the thumbnail image data and then doubled, that is, the thumbnail resolution magnification with respect to the relay resolution is doubled. However, other magnifications may be used as long as the relay resolution is lower than the thumbnail resolution.

  In the present embodiment, an example in which the present invention is applied when encoding thumbnail image data has been described. However, the encoding unit 11 can be applied to arbitrary image data including captured image data itself. Even when there is a restriction on the compression performance, it is possible to further reduce the size of the encoded image data obtained when compression encoding is performed.

  The imaging apparatus according to the present embodiment has been described as recording image data as a still image and generating thumbnail image data from image data that is a still image. However, the image data is, for example, AVCHD (Advanced Video Code High Definition). It may be recorded as a moving image, and thumbnail image data may be generated from a series of moving images. In that case, for example, the top image immediately after the start of shooting is extracted from the image data as a moving image to obtain image data as a still image, from which thumbnail image data is generated as in the present embodiment, and AVCHD is generated. It may be recorded as thumbnail image data defined by the standard.

  A part of the configuration of the imaging apparatus according to this embodiment may be configured as an integrated circuit. For example, the integrated circuit 3 may include the memory controller 7, the YC generation unit 8, the resolution conversion unit 10, the encoding unit 11, and the decoding unit 17. The integrated circuit 3 may further include at least one of the buffer memory 2, the microcomputer 12, and the storage unit 16.

  Since the present invention can store image data in a predetermined size when compression-coding without adding a special processing unit, the camera function such as a digital still camera, a video camera, a surveillance camera, a camera-equipped mobile terminal, etc. It can be applied to electronic devices having

DESCRIPTION OF SYMBOLS 1 Image pick-up part 2 Buffer memory 3 Integrated circuit 4 Memory card 5 Gain control part 6 A / D conversion part 7 Memory controller 8 YC production | generation part 10 Resolution conversion part 11 Encoding part 12 Microcomputer 13 Operation part 15 Display part 16 Storage part 17 Decoding Chemical department

Claims (7)

Imaging means for generating first image data having a first resolution;
Resolution conversion means for enlarging or reducing image data to an arbitrary resolution;
Encoding means for compressing and encoding image data to obtain encoded image data;
Control means for controlling the imaging means, the resolution converting means and the encoding means;
With
The control means controls the resolution conversion means to reduce the first image data to a second resolution lower than the first resolution to obtain second image data. One of the data and the image data obtained by enlarging or reducing the first image data by the resolution conversion means is temporarily reduced to a third resolution lower than the second resolution, and then enlarged to the second resolution. Then, a reduction / enlargement process for obtaining the second image data is performed,
The image pickup apparatus, wherein the control unit causes the encoding unit to encode the second image data to obtain encoded image data. The control means controls the resolution conversion means to reduce the first image data to a second resolution lower than the first resolution to obtain second image data. 2. The reduction / enlargement process is performed when the second image data obtained by reducing the data to the second resolution does not become a predetermined size or less when compressed by the encoding means. The imaging device described. The control means resets the third resolution to a lower resolution and performs the reduction / enlargement process when the second image data is not smaller than the predetermined size when the second image data is obtained by the reduction / enlargement process. The imaging device according to claim 1. Resolution conversion means for enlarging or reducing image data to an arbitrary resolution;
Encoding means for compressing and encoding image data to obtain encoded image data;
Control means for controlling the resolution conversion means and the encoding means;
With
The control means controls the resolution conversion means for the first image data, which is the first resolution input from the outside, to reduce the first image data to a second resolution lower than the first resolution. When obtaining the second image data, a third lower one of the first image data and the image data obtained by enlarging or reducing the first image data by the resolution conversion means is lower than the second resolution. Once the image is reduced to the resolution, the image is enlarged to the second resolution and the second image data is obtained.
The image processing apparatus, wherein the control unit causes the encoding unit to encode the second image data to obtain encoded image data. The control means controls the resolution conversion means for the first image data, which is the first resolution input from the outside, to reduce the first image data to a second resolution lower than the first resolution. When obtaining the second image data, the second image data obtained by reducing the first image data to the second resolution is not smaller than a predetermined size when compression-encoded by the encoding means. The image processing apparatus according to claim 4, wherein the reduction / enlargement process is performed. The control means resets the third resolution to a lower resolution and performs the reduction / enlargement process when the second image data is not smaller than the predetermined size when the second image data is obtained by the reduction / enlargement process. The image processing apparatus according to claim 4. An image for obtaining encoded image data by compressing and encoding the first image data, which is the first resolution input from the outside, to a second resolution lower than the first resolution. A processing method,
Step of reducing one of the first image data and image data obtained by enlarging or reducing the first image data to a third resolution lower than the second resolution to obtain third image data When,
Enlarging the third image data to the second resolution to obtain the second image data;
Compressing and encoding the second image data to obtain the encoded image data;
An image processing method comprising:

Images