JP2006211323A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor which is used for a digital still camera, can be prevented from increasing in circuit scale and development cost by restraining the amount of memories arranged in an LSI from increasing, and is capable of coping flexibly with an increase in the number of pixels of an image. <P>SOLUTION: Image data from a camera are introduced into a large-scale memory 11, the image data of one frame of the large-scale memory 11 are considered as the image data where M small frames are arranged in a horizontal direction, and the image data are read out by the small frame and subjected to image processing. The encoded data of each small frame are written in the large-scale memory 11 and read out from the large-scale memory 11. The capacity of a line memory used for interpolation processing and block conversion processing inside an image processing device is set corresponding to the lines of the small frames, so that the amount of the memories used can be restrained from increasing. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus suitable for use in a digital still camera or the like, and more particularly to reduction in hardware scale and development cost when the number of pixels of an image increases.

  2. Description of the Related Art Conventionally, for example, an encoding / decoding method based on the JPEG (Joint Photographic Experts Group) standard is known as a variable-length encoding method for efficiently encoding image data. In this system, a block of (8 × 8) pixels is used as an input / output unit in both cases where image data is encoded and decoded.

  For example, when image data is encoded by an image processing apparatus equipped with this method, the image data is input to a JPEG encoding unit in units of (8 × 8) pixel blocks and encoded. The encoded image data is sequentially stored in a recording medium. When image data is decoded by this image processing apparatus, the image data stored in the recording medium is read out, decoded by the JPEG decoding unit, and the image data is divided into blocks of (8 × 8) pixels. Output.

  On the other hand, many image input devices such as cameras and image output devices such as displays input and output image data in raster units. In raster scanning, scanning is performed from left to right. When scanning of one line is completed, the scanning proceeds to the lower line, and scanning is performed from left to right. Thereafter, scanning is repeatedly performed in order from left to right and from top to bottom. . The raster unit image data is input / output in the order of raster scanning.

  Therefore, when data is transmitted / received between a device in which such a processing unit is a raster unit and a device in which a processing unit such as image processing based on the JPEG standard is a block unit, as shown in Patent Document 1 In addition, it is necessary to temporarily store data in raster units (or block units) in a memory and perform conversion in data transfer units (hereinafter referred to as raster-block conversion) to be output from the memory in block units (or raster units). .

  FIG. 24 shows an example of a conventional image processing apparatus 100. In FIG. 24, image data is input by an image data input device such as a camera (not shown). The input image data is image data of a single color (R (red), G (green), or B (blue)) arranged in a Bayer array, and an image of one frame is, for example, (2564 (horizontal direction) × 1924 (vertical direction) pixels. The input image data is stored in the large capacity memory 101 via the bus 110. The input image data is written into the large capacity memory 101 in the order of raster scanning.

  The image data once stored in the large-capacity memory 101 is read in the order of raster scanning as in the writing, and is read to the data reading unit 103 via the bus 110. The image data read by the data reading unit 103 is sent to the preprocessing unit 104. The preprocessing unit 104 performs image processing for generating a suitable image, such as color interpolation processing and contour enhancement processing. The pre-processing unit 104 converts single-color image data into RGB image data by interpolation processing, and further converts RGB image data into YUV (luminance signal Y and color difference signals U and V) image data.

  The YUV image data from the preprocessing unit 104 is sent to the block conversion unit 105. The block conversion unit 105 performs processing (raster-block conversion) for converting the image data format in raster units into the data format in block units. For example, Y, U, and V image data output in raster units from the pre-processing unit 104 are stored in the block conversion memory for 8 lines and output in units of (8 × 8) pixel blocks for the subsequent stage. Is done.

The image data block-converted by the block converter 105 is sent to the encoder 106. The encoding unit 106 encodes the block unit image data input from the block conversion unit 105. For example, when compression encoding is performed by the JPEG method, DCT (Discrete Cosine Transform) calculation is performed for each block, and the data in the time domain is converted into the data in the frequency domain. Data converted to the frequency domain by DCT transform is quantized by a quantizer and is variable-length encoded by a Huffman code. The encoded data from the encoding unit 106 is sent to the large capacity memory 101 by the encoded data writing unit 108. Encoded data is stored in the large-capacity memory 101. The encoded data stored in the large-capacity memory 101 is read by the encoded data reading unit 109 and output to the subsequent stage.
Japanese Patent Laid-Open No. 11-41503

  When the image processing apparatus 100 as described above is mounted on a digital still camera, the electrical unit of the image processing apparatus 100 is often realized by LSI (Large Scale Integration). For example, an SDRAM (Synchronous Dynamic Random Access Memory) is used as the large capacity memory 101 of the image processing apparatus 100, and this large capacity memory 101 is generally arranged outside the LSI.

  In contrast, the color interpolation processing of the preprocessing unit 104 and the memory used in the block conversion unit 105 are often arranged in the middle of the image processing process. Since the number of terminals of the LSI is increased, it is generally provided inside the LSI.

  As the memory used in the preprocessing unit 104 and the block conversion unit 105, the following memory amount is required.

  In the pre-processing unit 104, the single-color image data of each pixel is converted into image data in which each pixel has three types of color information of R, G, and B by interpolation processing. For example, for example, a (5 × 5) filter that interpolates a color that does not exist in the target pixel from pixels near the target pixel is used. In order to perform the (5 × 5) filter processing, four line memory amounts are required.

  In the block conversion unit 105, for example, when the JPEG encoding process installed in the encoding unit 106 requires YUV (4: 2: 0), the color difference image data U and V are per line. Since the number of pixels is halved, eight line memories are required for the luminance image data Y, and four line memories are required for the color difference image data U and V, respectively. Furthermore, these block conversion memories require twice as much memory for writing and reading so as not to reduce the throughput. Accordingly, the image processing apparatus 100 requires 36 line memories in addition to the large capacity memory 101.

  However, with an increase in the number of pixels of an image, an image in which the number of pixels per line exceeds several thousand pixels is increasing. For this reason, if such a large number of line memories are provided inside the LSI, there is a problem that the circuit scale and development cost of the LSI increase.

  Further, in the conventional image processing apparatus 100, since the line memory provided in the LSI depends on the number of pixels per line, the image exceeding the number of pixels that can be stored in the line memory cannot be handled, and the number of pixels is small. If it changes, a design change is necessary.

  In view of the above-described problems, the present invention reduces the circuit size and development cost by suppressing the amount of memory used in an LSI, and can flexibly cope with an increase in the number of pixels of an image. An object is to provide a processing apparatus.

  In order to solve the above problem, an image processing apparatus according to a first aspect of the present invention includes a memory for storing image data, and M pieces of image data for one frame stored in the memory in a horizontal direction (M is It is regarded as image data in which small frames of (positive integers) are arranged, and the data reading unit for reading the small frames in raster scanning order and the image data read by the data reading unit are encoded with a predetermined number of lines as processing lines. An encoding unit, a code amount storage unit that sequentially stores a code amount of encoded data output from the encoding unit, and a code amount related to a processing line corresponding to each small frame stored in the code amount storage unit Based on this, an address for storing the encoded data in the memory is calculated, and the encoded data writing unit for storing the encoded data in the calculated address of the memory and the code amount storage unit are stored. Encoded data for calculating an address for reading out the encoded data stored in the memory based on the code amount associated with the corresponding processing line of each small frame, and reading out the encoded data from the calculated address in the memory And a reading unit.

  According to a second aspect of the present invention, in the first aspect of the present invention, the code amount storage unit stores, for each processing line, a code amount obtained by adding the code amount related to the processing line corresponding to each small frame. It is characterized by doing.

  According to a third aspect of the present invention, in the image processing apparatus according to the first aspect, the code amount storage unit stores the code amount for each processing line of each small frame.

  According to the image processing apparatus of the present invention according to claim 1, image data of one frame is regarded as image data in which M small frames are arranged in the horizontal direction, and image processing is performed for each small frame. The amount of memory used in the processing device can be reduced. Therefore, the scale and development cost of the image processing apparatus can be reduced. Further, even if the number of pixels of the image increases, it is possible to cope with the problem by adjusting the number of small frames without greatly affecting the memory configuration inside the image processing apparatus. Therefore, it is possible to provide an image processing apparatus that can flexibly cope with an increase in the number of pixels of an image.

  According to the image processing apparatus of the present invention according to claim 2, the code amount obtained by adding the code amount related to the processing line corresponding to each small frame is stored for each processing line, so that the code of each processing line is stored. Data can be read continuously.

  In addition, according to the image processing apparatus of the present invention according to claim 3, by including a code amount memory for storing a code amount for one frame for each processing line in a small frame, encoded data for one frame of the image. It is possible to extract a part of an image and store the encoded data without decoding the image, and to display only a part of the image at high speed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a main part of an example of an image processing apparatus to which the present invention is applied. As shown in FIG. 1, an image processing apparatus 1 to which the present invention is applied includes a large-capacity memory 11, an address storage unit 12, a data reading unit 13, a preprocessing unit 14, a block conversion unit 15, a code The encoding unit 16, the code amount storage unit 17, the encoded data writing unit 18, and the encoded data reading unit 19 are provided.

  The large-capacity memory 11 temporarily stores input image data and temporarily stores encoded data. The large-capacity memory 11 has an input image storage area 21 for storing input image data and an encoded data storage area 22 for storing encoded data.

  The input image storage area 21 of the large-capacity memory 11 has an image area for at least one frame. When writing image data into the input image storage area 21 of the large capacity memory 11, writing is performed in the order of raster scanning. When reading image data from the large-capacity memory 11, the image data of one frame of the large-capacity memory 11 is regarded as image data in which M small frames are arranged in the horizontal direction, and reading is performed in raster scanning order for each small frame. Do.

  The address storage unit 12 includes a register that stores an address of encoded data stored in the large-capacity memory 11. This register may be realized by a memory. Further, the address storage unit 12 itself may be provided outside the image processing apparatus 1.

  The data reading unit 13 regards one frame of image data stored in the input image storage area 21 of the large-capacity memory 11 as image data in which M (M is a positive integer) small frames are arranged in the horizontal direction. Image data is read from the large-capacity memory 11 in the order of raster scanning for each small frame. At this time, an address for reading the image data stored in the input image storage area 21 of the large-capacity memory 11 is calculated based on the corresponding register value stored in the address storage unit 12.

  The preprocessing unit 14 performs image processing for generating a suitable image, such as color interpolation processing and contour enhancement processing. The pre-processing unit 14 converts single-color (R, G, or B) image data into RGB image data through interpolation processing, and further converts into YUV image data.

  The block conversion unit 15 includes a block conversion memory for performing raster-block conversion, converts the raster-unit image data format input from the previous stage into a block-unit data format, and outputs it to the subsequent stage. For example, in the image processing apparatus 1 that performs JPEG encoding processing, image data for each of Y, U, and V output from the preprocessing unit 14 in units of rasters is stored in the block conversion memory for eight lines, and ( YUV image data is output for each block unit of 8 × 8) pixels. The block conversion unit 15 includes two types of block conversion memories for writing and reading so as not to cause a decrease in throughput due to raster-block conversion.

  The encoding unit 16 encodes the block-unit image data input from the block conversion unit 15 using, for example, the JPEG method, and outputs the encoded data to the subsequent stage. In JPEG encoding, image data is subjected to DCT conversion in units of blocks, and time-domain image data is converted to a frequency domain. Then, the DCT-converted image data is quantized and subjected to variable length coding using a Huffman code.

  The code amount storage unit 17 includes a code amount memory 31. When the encoding unit 16 encodes image data, the code amount storage unit 17 acquires the code amount from the encoding unit 16, and the code amount memory 31 of the code amount storage unit 17. Further, the code amount for each processing line is stored.

  The encoded data writing unit 18 sends the image data encoded by the encoding unit 16 to the large capacity memory 11 and stores it in the encoded data storage area 22 of the large capacity memory 11. The encoded data storage area 22 of the large capacity memory 11 is divided into storage areas for each processing line, and the encoded image data is stored in the encoded data storage area 22 of the large capacity memory 11 for each processing line. It will be done. At this time, the address for storing the encoded data in the encoded data storage area 22 of the large-capacity memory 11 is based on the address stored in the address storage unit 12 and the code amount stored in the code amount storage unit 17. Is calculated.

  The encoded data reading unit 19 reads the encoded data stored in the encoded data storage area 22 of the large-capacity memory 11 continuously for each processing line, reads the effective encoded data of each processing line, Output. At this time, the address for reading the encoded data from the encoded data storage area 22 of the large capacity memory 11 is based on the address stored in the address storage unit 12 and the code amount stored in the code amount storage unit 17. Calculated.

  Next, the operation of the image processing apparatus configured as described above will be described. In FIG. 1, image data is input by an image data input device such as a camera (not shown). The input image data is sent to the large-capacity memory 11 via the bus 10. The input image data is written in the input image storage area 21 of the large capacity memory 11 in the order of raster scanning. As shown in FIG. 2, the image data is single-color image data in which R, G, and B pixels are arranged in a Bayer array. The number of pixels in one frame is, for example, (2564 (horizontal direction) × 1924 (vertical). Direction)).

  The image data once accumulated in the input image storage area 21 of the large capacity memory 11 is read out to the data reading unit 13 via the bus 10. At this time, one frame of image data stored in the input image storage area 21 of the large-capacity memory 11 is regarded as image data in which M small frames are arranged in the horizontal direction. Data is read out.

  That is, FIG. 3 and FIG. 4 illustrate the transfer of data from the input image storage area 21 of the large capacity memory 11. As shown in FIG. 3, image data of (2564 pixels × 1924 pixels) is written in the input image storage area 21 of the large capacity memory 11 in the order of raster scanning.

  That is, the image data of the uppermost line “1” of one frame is written in the horizontal direction for 2564 pixels from left to right. Thereafter, image data is written in the order of raster scanning from line “1” to line “1924” for each line.

  When image data is read from the input image storage area 21 of the large-capacity memory 11, one frame is divided into M small frames as shown in FIG. In this example, one frame is divided into four small frames A, B, C, and D. Then, image data is read out in the order of raster scanning for each of the small frames A to D.

  That is, first, the image data of the line “1” of the small frame A is read from the left to the right for 644 pixels in the horizontal direction. Hereinafter, image data is read out in the order of raster scanning for each line from line “1” to line “1924”.

  When the reading of the small frame A is completed, the image data of the small frame B is then read from the line “1” to the line “1924” in the order of raster scanning. When the reading of the small frame B is completed, the image data of the small frame C is then read in the order of raster scanning from the line “1” to the line “1924”, and when the reading of the small frame C is completed, the small frame D is read. Image data is read from line “1” to line “1924” in the order of raster scanning.

  At this time, an address for reading the image data stored in the input image storage area 21 of the large-capacity memory 11 is calculated based on the corresponding register value stored in the address storage unit 12.

  When the image is read out in this way, as shown in FIG. 4, there are pixel portions Pa to Pc that are read out redundantly in adjacent small frames. The reason why there are pixel portions Pa to Pc that are redundantly read out will be described later.

  In this example, one frame is divided equally and the number of pixels in the horizontal direction of all the small frames A to D is 644 pixels. However, the number of pixels may be different for each small frame. Here, one frame is divided into four small frames, but the present invention is not limited to this.

  In FIG. 1, the image data read by the data reading unit 13 in the order of small frames A to D is sent to the preprocessing unit 14. The preprocessing unit 14 performs image processing for generating a suitable image, such as color interpolation processing and contour enhancement processing. The image data subjected to the color interpolation processing is changed from image data of a single color with one pixel being R, G, or B to image data having three types of color information with one pixel being R, G, B. For example, for example, a (5 × 5) interpolation filter that interpolates a color that does not exist in the pixel of interest from pixels in the vicinity of the pixel of interest is used. Pixel data having RGB information for each pixel by interpolation processing is converted into three types of image data of Y, U, and V by matrix calculation.

  The filter process when performing such an interpolation process has a property that the image size after the filter process is smaller than the image size before the filter process. This is because, as shown in FIG. 5A, when there is valid reference pixel data around the pixel of interest Px, filter calculation can be performed using image data around the pixel of interest Px. As shown in (B), when the pixel of interest Py to be subjected to the filter calculation is near the edge of the image, the reference pixel of the (5 × 5) filter includes image data that is not valid, and the calculation result of that pixel Is invalid data.

  For example, when (5 × 5) filter processing is performed on a small frame having an image size of (644 pixels × 1924 pixels) as in the present embodiment, the effective image size after the filter processing is (640 pixels × 1920). Pixel). Therefore, it is necessary to determine the image size of the small frame in consideration of the image size that is reduced by the filtering process. This is also the reason why there are pixel portions Pa to Pc that are read out redundantly in adjacent small frames, as shown in FIG.

  In FIG. 1, the YUV image data from the preprocessing unit 14 is sent to the block conversion unit 15. The block conversion unit 15 performs processing for converting the raster-unit image data format into the block-unit data format. For example, when JPEG encoding processing is performed, 8 lines of Y, U, and V image data output from the preprocessing unit 14 in units of rasters are stored in the block conversion memory, and (8 X8) Output for each block of pixels.

  The image data block-converted by the block converter 15 is sent to the encoder 16. The encoding unit 16 encodes the block-unit image data input from the block conversion unit 15. For example, when compression encoding is performed by the JPEG method, DCT calculation is performed for each block, and data from the time domain is converted to data in the frequency domain. The data transformed into the frequency domain by DCT transformation is quantized by a quantizer and variable-length coded using a Huffman code.

  The code amount of the encoded data encoded by the encoding unit 16 is acquired by the code amount storage unit 17. The code amount storage unit 17 is provided with a code amount memory 31 for storing a code amount for each processing line. As shown in FIG. 6, the code amount memory 31 is provided with areas ARA_M1 to ARA_M240 for storing the code amount for each processing line.

  In the code amount storage unit 17, when the small frame encoded by the encoding unit 16 includes the left end of the image, the code amount of the processing line (hereinafter referred to as a domain) in the small frame is encoded for each processing line. They are stored in areas ARA_M1 to ARA_M240 of the quantity memory 31, respectively. When the small frame encoded by the encoding unit 16 does not include the left end of the image, the code amount of the corresponding domain and the code amount of the corresponding processing line already stored in the areas ARA_M1 to ARA_M240 of the code amount memory 31 are obtained. The addition result is stored in the areas ARA_M1 to ARA_M240 of the code amount memory 31 for each processing line as the code amount of the new corresponding processing line. As a result, the code amount for each processing line is stored in the areas ARA_M1 to ARA_M240 of the code amount memory 31 of the code amount storage unit 17, respectively.

  In this example, since encoding is performed using the JPEG method, which is a variable-length code, such a code amount storage unit 17 is required, but the encoding process in the encoding unit 16 is a fixed-length encoding process. Since the code amount can be estimated in advance, it is not necessary to include the code amount storage unit 17. Further, instead of storing the code amount, the step width of the address in the large-capacity memory 11 or the address itself may be calculated from the code amount and stored in the code amount storage unit 17.

  The code amount stored in the code amount storage unit 17 is encoded from the large-capacity memory 11 by, for example, the encoded data reading unit 19 described later until the encoded data stored in the large-capacity memory 11 becomes unnecessary. The data may be stored until it is read and stored in an external storage medium (not shown). Further, in the present embodiment, when image processing for a plurality of frames is continuously executed and encoded data for a plurality of frames is stored in the large-capacity memory 11, a code amount memory 31 having the same number as the number of frames is provided. Just do it. The detailed description of the code amount storage unit 17 will be made together with the description of the encoded data writing unit 18 below.

  In FIG. 1, the encoded data from the encoding unit 16 is sent to the large capacity memory 11 by the encoded data writing unit 18. As shown in FIG. 7, the encoded data storage area 22 of the large-capacity memory 11 is divided into storage areas for each processing line, and includes memory areas ARA_N1 to ARA_N240 for each processing line. The encoded data writing unit 18 stores the encoded data of at least one domain constituting the small frame in the memory areas ARA_N1 to ARA_N240 of the processing line to which the domain belongs.

  At this time, if the domain corresponding to the stored encoded data includes the left end of the image, the encoded data is stored from the top of the memory areas ARA_N1 to ARA_N240 of the processing line to which the corresponding domain belongs. On the other hand, when the domain corresponding to the stored encoded data does not include the left end of the image, in each memory area ARA_N1 to ARA_N240, immediately after the encoded data corresponding to the same processing line of the small frame storing the code amount immediately before From this, the encoded data is stored.

  That is, the address storage unit 12 stores the top addresses ADDR1 to ADDR240 of the memory areas ARA_N1 to ARA_N240. When the domain corresponding to the stored encoded data includes the left end of the image, the encoded data is referred to from the top of each of the memory areas ARA_N1 to ARA_N240 with reference to the start addresses ADDR1 to ADDR240 stored in the address storage unit 12. Stored.

  Further, the code amount memory 31 of the encoded data writing unit 18 stores the code amount of the encoded data of the same processing line stored so far. When the domain corresponding to the encoded data to be stored does not include the left end of the image, the start address ADDR1 to ADDR240 of the memory area ARA_N1 to ARA_N240 corresponds to the code amount from the code amount memory 31 of the encoded data writing unit 18. Add the addresses to be used. By storing the encoded data from the address calculated in this way, in each of the memory areas ARA_N1 to ARA_N240, the code is encoded immediately after the encoded data corresponding to the same processing line of the small frame in which the code amount is stored immediately before. Stored data.

  FIG. 8 illustrates the configuration of a small frame, a processing line, and a domain when performing an encoding process. When the encoding unit 16 performs the encoding process, in FIG. 8, first, the encoding process of the processing line “1” (domain A1) of the small frame A is performed. When (8 × 8) blocking is performed, the processing line corresponds to pixel data for 8 lines. Next, the encoding process of the processing line “2” (domain A2) of the small frame A is performed, and the encoding process of the processing line “3” (domain A3) of the small frame A is performed. The encoding process is sequentially performed up to the process line “240” (domain A240).

  As shown in FIG. 7, the encoded data storage area 22 of the large-capacity memory 11 is divided into memory areas ARA_N1 to ARA_N240 for each line, and the encoded data of each domain is the processing line to which that domain belongs. Are stored in the memory areas ARA_N1 to ARA_N240, respectively. At this time, if the domain corresponding to the stored encoded data includes the left end of the image, the encoded data is stored from the top of the memory areas ARA_N1 to ARA_N240 of the processing line to which the corresponding domain belongs. Since the image of the small frame A includes the left end, the encoded data ENC_A1 to ENC_A240 of each domain A1 to A240 is stored from the top of the memory areas ARA_N1 to ARA_N240 of the processing line to which the domain belongs. The start addresses ADDR1 to ADDR240 of the memory areas ARA_N1 to ARA_N240 are referred to from the address storage unit 12.

  Thus, when encoding each processing line “1”, “2”, “3”,..., “240” (domains A1, A2, A3,..., A240) of the small frame A, The code amount is acquired by the code amount storage unit 17. As described above, in the code amount storage unit 17, when the small frame encoded by the encoding unit 16 includes the left end of the image, the code amount of the processing line (domain) in the small frame is set for each region of each processing line. Is stored in the code amount memory 31. Since the image of the small frame A includes the left end, the code amount of each processing line (domain) in the small frame A is stored in the code amount memory 31 of the code amount storage unit 17 for each region of each processing line. The

  That is, the encoded data ENC_A1 of the domain A1 encoded by the encoding unit 16 is stored from the beginning of the memory area ARA_N1 of the encoded data storage area 22, as shown in FIG. When the encoding of the domain A1 is completed, the code amount AMT_A1 of the domain A1 is acquired by the code amount storage unit 17. As shown in FIG. 10, the code amount AMT_A1 is stored in the area ARA_M1 of the code amount memory 31 as the code amount SUM1 (SUM1 = AMT_A1) of the processing line “1”.

  Next, the encoded data ENC_A2 of the domain A2 is stored from the beginning of the memory area ARA_N2 of the encoded data storage area 22, as shown in FIG. When the encoding of the domain A2 is completed, the code amount AMT_A2 of the domain A2 is acquired by the code amount storage unit 17. The code amount AMT_A2 is stored in the area ARA_M2 of the code amount memory 31 as the code amount SUM2 (SUM2 = AMT_A2) of the processing line “2” as shown in FIG.

  Hereinafter, the encoded data ENC_A1 to ENC_A240 of the domains A1 to A240 of the small frame A are respectively stored from the top of the memory areas ARA_N1 to ARA_N240 of the encoded data storage area 22, as shown in FIG. When the encoding of the domains A1 to A240 of the small frame A is completed, the code amounts AMT_A1 to AMT_A240 of the domains are acquired by the code amount storage unit 17, and the code amounts AMT_A1 to AMT_A240 are used as the code amounts of the respective processing lines. As shown in FIG. 10, the data is stored in areas ARA_M <b> 1 to ARA_M <b> 240 of the code amount memory 31.

  When the encoding of the small frame A is completed, the encoding process of the processing line “1” (domain B1) of the small frame B is performed, and then the processing line “240” (domain B240) of the small frame B is performed. In order, the encoding process is performed. Since the image of the small frame B does not include the left end, the encoded data of each domain is immediately after the encoded data corresponding to the same processing line of the small frame storing the code amount immediately before in each memory area ARA_N1 to ARA_N240. From this, the encoded data is stored. At this time, the code amount SUM1 to SUM240 for each processing line of the code amount memory 31 of the code amount storage unit 17 is added to the top addresses ADDR1 to ADDR240 of the memory areas ARA_N1 to ARA_N240 stored in the address storage unit 12. It can be obtained by adding.

  In this way, when encoding is performed on each processing line “1”, “2”, “3”,..., “240” (domains B1, B2, B3,..., B240) of the small frame B, The code amount is acquired by the code amount storage unit 17. Since the small frame B does not include the left end of the image, the code amount of the corresponding domain and the code amount of the corresponding processing line already stored in the code amount memory 31 are added, and the addition result is a new code amount of the corresponding processing line. Is stored in the code amount memory 31 for each processing line.

  That is, the encoded data ENC_B1 of the domain B1 encoded by the encoding unit 16 is stored subsequent to the encoded data ENC_A1 in the memory area ARA_N1 of the encoded data storage area 22, as shown in FIG. When the encoding of the domain B1 is completed, the code amount AMT_B1 of the domain B1 is acquired by the code amount storage unit 17. This code amount AMT_B1 is added to the previous code amount SUM1 of the processing line “1”, and the addition result is a new code amount of the processing line “1”, as shown in FIG. 12, in the area ARA_M1 of the code amount memory 31. (SUM1 = SUM1 + AMT_B1). Since the area ARA_M1 stores the code amount AMT_A1 in the domain A1 so far, the code amount SUM1 of the processing line “1” is (SUM1 = AMT_A1 + AMT_B1).

  The encoded data ENC_B2 of the domain B2 encoded by the encoding unit 16 is stored subsequent to the encoded data ENC_A2 in the memory area ARA_N2 of the encoded data storage area 22, as shown in FIG. When the encoding of the domain B2 is completed, the code amount AMT_B2 of the domain B2 is acquired by the code amount storage unit 17. This code amount AMT_B2 is added to the previous code amount SUM2 of the processing line “2”, and the addition result is a new code amount of the processing line “2”, as shown in FIG. 12, in the area ARA_M2 of the code amount memory 31. (SUM2 = SUM2 + AMT_B2). Since the code amount AMT_A2 in the domain A2 is stored in the area ARA_M2 until then, the code amount SUM2 of the processing line “2” is (SUM2 = AMT_A2 + AMT_B2).

  Hereinafter, the encoded data ENC_B1 to ENC_B240 of each of the domains B1 to B240 of the small frame B follow the encoded data ENC_A1 to ENC_A240, following the memory areas ARA_N1 to ARA_N240 of the encoded data storage area 22, as shown in FIG. Each is remembered. When the encoding of the domains B1 to B240 of the small frame B is completed, the code amounts AMT_B1 to AMT_B240 of the domains are acquired by the code amount storage unit 17, and the code amounts AMT_B1 to AMT_B240 are used as the code amounts of the respective processing lines. 12, the data is stored in areas ARA_M1 to ARA_M240 of the code amount memory 31, respectively.

  When the encoding of the small frame B is completed, the encoding process of the small frame C is performed next. As shown in FIG. 13, the encoded data ENC_C1 to ENC_C240 of the domains C1 to C240 of the small frame C are stored in the memory areas ARA_N1 to ARA_N240 of the encoded data storage area 22 and the encoded data ENC_B1 to ENC_B240, respectively. Is done. When the encoding of the domains C1 to C240 of the small frame C is completed, the code amounts AMT_C1 to AMT_C240 of the domains are acquired by the code amount storage unit 17, and the code amounts AMT_C1 to AMT_C240 are used as the code amounts of the respective processing lines. 14, the data is stored in the areas ARA_C1 to ARA_C240 of the code amount memory 31, respectively.

  When the encoding of the small frame C is completed, the encoding process of the small frame D is performed next. The encoded data ENC_D1 to ENC_D240 of the domains D1 to D240 of the small frame D are stored in the memory areas ARA_N1 to ARA_N240 of the encoded data storage area 22 after the encoded data ENC_C1 to ENC_C240, as shown in FIG. The When encoding of the domains D1 to D240 of the small frame D is completed, the code amounts AMT_D1 to AMT_D240 of the domains are acquired by the code amount storage unit 17, and the code amounts AMT_D1 to AMT_D240 are used as the code amounts of the respective processing lines. As shown in FIG. 16, it is stored in the areas ARA_C1 to ARA_C240 of the code amount memory 31, respectively.

  When the encoding process from the small frames A to D is completed as described above, the areas ARA_N1 to ARA_N240 of the encoded data storage area 22 of the large-capacity memory 11 are set for each processing line as shown in FIG. Encoded data ENC_A1 to ENC_D1, ENC_A2 to ENC_D2,..., ENC_A240 to ENC_D240 are stored. Further, as shown in FIG. 16, the code amount storage unit 17 stores code amounts SUM1 to SUM240 for each processing line.

  In FIG. 1, the encoded data once stored in the encoded data storage area 22 of the large-capacity memory 11 as described above is read out for each processing line by the encoded data reading unit 19. At this time, the address that the encoded data reading unit 19 issues to the large-capacity memory 11 is stored in the start address of the memory areas ARA_N1 to ARA_N240 of each processing line stored in the address storage unit 12 and the code amount storage unit 17. It is calculated from the code amount SUM1 to SUM240 of each processed line.

  That is, first, the start address ADDR1 of the memory area ARA_N1 of the processing line “1” in the memory space of the large capacity memory 11 in FIG. 15 is referred from the address storage unit 12, and the processing line “1” of the processing line “1” is used by using the address ADDR1. The read start position of the encoded data of the encoded data is set. Then, the code amount SUM1 of the processing line “1” is referred from the area ARA_M1 of the code amount storage unit 17 in FIG. 16, and an address corresponding to the code amount corresponding to the code amount SUM1 is calculated and calculated from the address ADDR1. Data is read by the address. As a result, the encoded data (ENC_A1, ENC_B1, ENC_C1, ENC_D1) of the processing line “1” is read from the memory area ARA_N1 of the encoded data storage area 22 of the large capacity memory 11, and the read encoded data is read. Is output to the subsequent stage.

  When the reading of the encoded data of the processing line “1” is completed, the encoded data of the processing line “2” is continuously read, and the read encoded data is output to the subsequent stage. Reading of the encoded data of the processing line “2” is performed in the same manner as the processing line “1”.

  In this way, until the processing line “240”, the reading of the encoded data and the output of the encoded data to the subsequent stage are repeatedly performed from the memory areas ARA_N1 to ARA_N240 of the encoded data storage area 22 of the large capacity memory 11. Is called.

  In the present embodiment, the encoded data reading unit 19 reads the encoded data from the first processing line. However, the present invention is not limited to this, and data can be read from any processing line.

  As described above, as shown in FIG. 17, the encoded data reading unit 19 continuously outputs the encoded data of the processing lines “1”, “2”, “3”,. The encoded data output from the encoded data reading unit 19 is not different from the order of the encoded data generated by the general JPEG encoding process. Therefore, no special decoding method is required for decoding the encoded data, and the encoded data can be decoded by a general JPEG decoding method.

  Note that when the encoded data to be decoded is transferred in the image processing apparatus of the present invention, it is not always necessary to transfer the data without gaps as shown in FIG. In this case, when the transferred encoded data is transferred again for decoding, the code amount of each processing line is referred to and the data may be transferred to the decoding device without any gap. Therefore, in this case, it is necessary to hold the code amount of each processing line until the encoded data is decoded or to transfer the code amount together with the encoded data.

  Further, as a device provided in the subsequent stage of the encoded data reading unit 19, for example, an external storage device or a communication device can be considered, but the device is not limited to this. In addition, these devices may be provided so as to use the encoded data reading unit 19 exclusively, or may be provided in parallel by including a plurality of encoded data reading units 19.

  In the above-described embodiment, JPEG is used as the encoding method. However, the present invention is not limited to the JPEG method, and can be similarly applied to other encoding methods. In the above example, the number of pixels is (2564 × 1924), but the present invention can be applied to any number of pixels.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, in the above-described embodiment, as shown in FIG. 6, the code amount storage unit 17 stores the code amount for each processing line of one frame. However, as shown in FIG. May be stored. That is, the code amount storage unit 17 includes a code amount memory that stores the code amount of one frame for each domain, and is configured to store the code amount of each domain in a corresponding area of the code amount memory. In this case, the large-capacity memory 11, the encoded data writing unit 18, and the encoded data reading unit 19 are modified as follows.

  In the large-capacity memory 11, the encoded data storage area for at least one frame is divided into the number of domains constituting one frame.

  The encoded data writing unit 18 stores the encoded data of each domain for each corresponding memory area in the large capacity memory 11. At this time, the address that the encoded data writing unit 18 issues to the large-capacity memory 11 is stored in the start address of the memory area of each domain stored in the address storage unit 12 and the code amount storage unit 17. It is calculated from the code amount of each block.

  For example, as in the above-described embodiment, when one frame is divided into four small frames and the unit of the processing line is 8 lines, the code amount storage unit 17 performs each domain as shown in FIG. The encoded data is stored, and the encoded data writing unit 18 stores the encoded data for each domain in the encoded data storage area 22 of the large capacity memory 11 as shown in FIG.

  The encoded data reading unit 19 continuously reads the encoded data stored in the large capacity memory 11 for each domain, and outputs only the effective encoded data of each domain to the subsequent stage. At this time, the address that the encoded data reading unit 19 issues to the large-capacity memory 11 is the start address of the memory area of each domain stored in the address storage unit 12 and each address stored in the code amount storage unit 17. Calculated from the code amount of the domain. Note that the encoded data transfer method is the same as the transfer method before the transformation.

  In such a modification, for example, a part of the image is extracted and stored without decoding the encoded data for one frame of the image, and only a part of the image is displayed at high speed. Is possible.

  As described above, according to the embodiment of the present invention, one frame of image data is regarded as image data in which M small frames are arranged in the horizontal direction, and image processing is executed for each small frame. ing. As a result, the amount of memory used in the LSI of the image processing apparatus can be suppressed, and the hardware scale and development cost of the image processing apparatus can be reduced. This will be described in detail.

  As described above, the preprocessing unit 14 in FIG. 1 uses an interpolation filter to interpolate colors that do not exist in the pixel of interest with pixels near the pixel of interest. For example, an interpolation filter of (5 × 5) is used for the interpolation. Here, in order to simplify the explanation, the interpolation filter of (3 × 3) will be used.

  For example, in FIG. 20, if the pixel P5 is the target pixel, the image data of the target pixel P5 has only the B color information, and does not have the R color information or the G color information. For this reason, R color information and G color information are formed by interpolation using pixels around the pixel of interest P5.

That is, the R color information of the target pixel P5 is obtained by the pixels P1, P3, P7, and P9 of the pixel R around the target pixel P5.
H1 × P1 + H2 × P3 + H3 × P7 + H4 × P9
It is calculated | required by the filter calculation. The G color information of the target pixel P5 is obtained by the pixels P2, P4, P6, and P8 of the pixels G around the target pixel P5.
H1 × P2 + H2 × P4 + H3 × P6 + H4 × P8
It is calculated | required by the filter calculation. However, H1-H4 are filter coefficients.

  As shown in FIG. 21, the interpolation filter that performs the above-described filter calculation generally has 1-sample delay circuits 51a to 51f and 1-line delay circuits 52a and 52b for obtaining the output of each tap, and filter coefficients. Multipliers 53a to 53i for multiplication and an adder 54 can be used. In FIG. 21, an unnecessary tap output may be obtained by giving a coefficient “0” to the multipliers 53a to 53i.

  Here, the one-line delay circuits 52a and 52b are composed of line memories. Conventionally, since image data is processed according to raster scanning, when the number of pixels is (2564 × 1924), as a line memory constituting the one-line delay circuits 52a and 52b, as shown in FIG. A capacity for pixels is required. On the other hand, in the embodiment of the present invention, one frame of image data is regarded as image data in which, for example, four small frames are arranged in the horizontal direction, and the image data is processed for each small frame. For this reason, as the line memory that constitutes the one-line delay circuits 52a and 52b, a capacity of 640 pixels is sufficient as shown in FIG. Therefore, the memory capacity in the color interpolation processing of the preprocessing unit 14 can be reduced to about ¼ compared with the conventional case.

  Further, in the block conversion unit 15 in FIG. 1, the block data of (8 × 8) pixels is formed by raster-block conversion, but here, for the sake of simplicity of description, (4 × 4) A case where a block is formed will be described.

  The conversion from image data in the order of raster scanning into (4 × 4) block data can be configured by four line memories 61a to 61d as shown in FIG.

  Here, conventionally, since image data is processed according to raster scanning, when the number of pixels is (2564 × 1924), the line memories 61a to 61d have capacity of 2564 pixels as shown in FIG. Is required. On the other hand, in the embodiment of the present invention, one frame of image data is regarded as image data in which, for example, four small frames are arranged in the horizontal direction, and the image data is processed for each small frame. Therefore, the line memories 61a to 61d only need to have a capacity of 640 pixels as shown in FIG. Therefore, the memory capacity in the block converter 15 can be reduced to about ¼ compared to the conventional case.

  Further, in the embodiment of the present invention, even if the number of pixels of an image increases, it is possible to cope with the problem by adjusting the number of small frames without greatly affecting the memory configuration inside the image processing apparatus. Therefore, it is possible to provide an image processing apparatus that can flexibly cope with an increase in the number of pixels of an image.

  Further, the present embodiment is modified to include a code amount memory for storing the code amount for one frame for each processing line in the small frame, so that the encoded data for one frame of the image is not decoded. It is possible to extract a part of the image and save the encoded data, and to display only a part of the image at high speed.

  The present invention relates to an image processing apparatus such as a digital still camera that converts raster unit image data to block unit image data or processes block unit image data to raster unit image data. Can be used.

1 is a block diagram illustrating a main part of an example of an image processing apparatus to which the present invention is applied. It is explanatory drawing of the image data of a Bayer arrangement. It is explanatory drawing of transfer of the image data in the image processing apparatus to which this invention was applied. It is explanatory drawing of transfer of the image data in the image processing apparatus to which this invention was applied. It is explanatory drawing of the filter process in the image processing apparatus to which this invention was applied. It is explanatory drawing of the code amount memory | storage part in the image processing apparatus to which this invention was applied. It is explanatory drawing of the encoding data storage area | region of the large capacity memory in the image processing apparatus to which this invention was applied. It is explanatory drawing of the small frame in the image processing apparatus to which this invention was applied, a processing line, and a domain. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is operation | movement explanatory drawing of the image processing apparatus to which this invention was applied. It is explanatory drawing of the data transfer in the image processing apparatus to which this invention was applied. It is explanatory drawing of the modification of the image processing apparatus to which this invention was applied. It is explanatory drawing of the modification of the image processing apparatus to which this invention was applied. It is explanatory drawing of the interpolation process of the pre-processing part in the image processing apparatus to which this invention was applied. It is a block diagram used for description of memory reduction in the image processing apparatus to which the present invention is applied. It is a block diagram used for description of memory reduction in the image processing apparatus to which the present invention is applied. It is explanatory drawing of the block conversion part in the image processing apparatus to which this invention was applied. It is a block diagram of an example of the conventional image processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Bus 11 Large capacity memory 12 Address storage part 13 Data reading part 14 Preprocessing part 15 Block conversion part 16 Encoding part 17 Code amount storage part 18 Encoded data writing part 19 Encoded data reading part

Claims (3)

A memory for storing image data;
A data reading unit that regards image data for one frame stored in the memory as image data in which M small frames (M is a positive integer) are arranged in a horizontal direction, and reads the small frames in raster scanning order;
An encoding unit that encodes the image data read by the data reading unit using a predetermined number of lines as processing lines;
A code amount storage unit for sequentially storing code amounts of encoded data output from the encoding unit;
Based on the code amount relating to the processing line corresponding to each small frame stored in the code amount storage unit, the address for storing the encoded data in the memory is calculated, and the calculated address in the memory is calculated. An encoded data writing unit for storing the encoded data;
Based on the code amount related to the processing line corresponding to each small frame stored in the code amount storage unit, the address for reading the encoded data stored in the memory is calculated, and the calculated address of the memory An image processing apparatus comprising: an encoded data reading unit that reads encoded data from the image data. The image processing apparatus according to claim 1, wherein the code amount storage unit stores, for each processing line, a code amount obtained by adding a code amount related to the processing line corresponding to each small frame. The image processing apparatus according to claim 1, wherein the code amount storage unit stores the code amount for each processing line of each small frame.

Images