JP2007158524A - Image coding apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image coding apparatus which hardly causes noises, and the scale of hardware of which is small, in the case of image compression, and which needs a reduced computational complexity, and to provide an image coding method. <P>SOLUTION: The image coding apparatus for a moving picture of a primary color filter image using image filters for extracting a plurality of pixel blocks from blocks each comprising N×N pixels of an image array by color components through the separation of the blocks by color components by using color filters and thereafter coding the pixel blocks is characterized by including: a motion detection means for applying motion detection to each of the pixel blocks in terms of 1×1 point and extracting a motion vector; and a compression means for compressing the moving picture on the basis of the motion vector. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image encoding apparatus and an image encoding method for compressing a moving image of an image filter primary color filter image.

Conventionally, various methods have been proposed for compressing a captured moving image in an apparatus for transmitting an image of a micro camera used for medical treatment such as a capsule endoscope. For example, there is a compression technique disclosed in Patent Document 1. In the compression technique disclosed in Patent Document 1, an image signal obtained from an image array is converted into a digital signal, and converted into a 4 × 4 pixel block for each color component by a 4 × 4 pixel RGB component extraction circuit. A two-dimensional discrete cosine transform is performed on each block by a circuit, and then quantization is performed by a quantization circuit. Thereafter, zero length compression is performed on the AC coefficient by the Huffman coding circuit, and variable length coding is performed on the non-zero AC coefficient and DC coefficient by the Huffman coding circuit.
JP 2005-217896 A

  However, the compression technology of the conventional image encoding device increases the scale of hardware such as two-dimensional discrete cosine transform and motion detection, and the amount of calculation is large. Therefore, it is mounted on a micro camera used for medical purposes. Difficult to do.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide an image encoding apparatus and an image encoding method that are less likely to generate noise during image compression, have a small hardware scale, and a small amount of calculation.

  The image coding apparatus according to claim 1, further comprising: a motion detection unit that performs motion detection on a pixel block at 1 × 1 point and extracts a motion vector; and a compression unit that compresses the moving image based on the motion vector. It is characterized by.

  The image coding apparatus according to claim 2 is characterized in that compression of motion compensated prediction coding is used as the compression means.

  The image coding apparatus according to claim 3 is characterized in that the motion detection means extracts a motion vector by taking a difference between a pixel of a compression target image and a reference image in the vicinity thereof.

  The image encoding apparatus according to claim 4 is characterized in that the motion vector is a motion vector of a macroblock itself within a predetermined range of the pixel block and a motion vector of a pixel in the macroblock.

  6. The image encoding apparatus according to claim 5, wherein the motion detection means moves simultaneously with respect to the color components of the pixel block of one red (R), two green (G1, G2), and one blue (B). The detection is performed.

  The image encoding method according to claim 6 is characterized in that motion detection is performed on a pixel block at 1 × 1 point to extract a motion vector, and the moving image is compressed based on the motion vector.

  The image encoding method according to claim 7 is characterized in that compression of motion compensated prediction encoding is used as compression.

  The image encoding method according to claim 8 is characterized in that the motion detection extracts a motion vector by taking a difference between a pixel of a compression target image and a reference image in the vicinity thereof.

  The image coding method according to claim 9 is characterized in that the motion vector is a motion vector of the macroblock itself within a predetermined range of the pixel block and a motion vector of the pixel in the macroblock.

  The image coding method according to claim 10, wherein the motion detection is performed simultaneously on the color components of one red (R), two green (G 1, G 2), and one blue (B) pixel block. It is characterized by performing.

  According to the present invention, the image is compressed by including a motion detection unit that performs motion detection on the pixel block at 1 × 1 point and extracts a motion vector, and a compression unit that compresses the moving image based on the motion vector. In addition, it is difficult to generate noise in image compression, the hardware scale is small, and the amount of calculation can be reduced.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. An image coding apparatus according to an embodiment of the present invention is an image filter primary color filter image that performs coding after extracting a plurality of pixel blocks separately for each color component from a block composed of N × N pixels of an image array using a color filter. The moving image is compressed.

  FIG. 1 is a block diagram showing an example of an image encoding device according to the present invention. FIG. 2 is an explanatory diagram of a color filter of the image coding apparatus. FIG. 3 is an explanatory diagram showing the operation of the image coding method. FIG. 4 is a block diagram showing the image encoding method. FIG. 5 is a flowchart showing the operation of the image encoding method. 6 to 12 are explanatory diagrams showing the operation of the image coding method.

As shown in FIG. 1, in the image coding apparatus of the present embodiment, a single-plate color type image sensor has a mosaic arrangement of color filters 5 on an imaging unit, and red (hereinafter R), Green (hereinafter G) and blue (hereinafter B) color signals are obtained. Specifically, the signal of the imaging unit 5 is output by the horizontal scanner 16 via the vertical scanner 12 and the noise cancellation circuit 14, digitized by the A / D converter 20 via the amplifier 18, and the buffer memory 24 and the prediction difference code are output. The output is made via the conversion circuit 26. Since only one color filter can be used for one pixel, each missing color is estimated by calculating the missing color by color interpolation processing (hereinafter, the interpolated image is a bitmap image). Called). In fact, separated from the N _H × N _V pixels consisting block of an image array using a color filter shown in FIG. 2 (a) for each color component, after extracting a plurality of pixel blocks shown in FIG. 2 (b) Encoding is performed.

  Next, the configuration and operation of the image coding apparatus according to the present embodiment will be described. As shown in FIG. 3, the output (a) of the Bayer color filter is rearranged as shown in (b), and 1 × 1 point motion detection is performed. If the result is normal, orthogonal transform (DCT) is not performed, but suddenly quantization is performed and variable length coding is performed. A block diagram is shown in FIG. The contents of FIGS. 3 and 4 will be described in detail with reference to other drawings.

  As shown in FIG. 1, the signals digitized by A / D conversion by the A / D converter 20 from the imaging unit 5 having the Bayer color filter are arranged as shown in FIG. Are rearranged as shown in FIG. 2B and read out to the buffer memory 24. This image signal is treated as a compression target image without undergoing interpolation processing or color conversion processing to the YUV color system.

  FIG. 5 shows a flowchart for performing motion compensation predictive coding (P picture in the standard compression method). First, in FIG. 5A, the entire screen is divided into macro blocks (S100), and 1 × 1 point motion detection is performed (S101). FIG. 5B shows the motion detection of 1 × 1 point in the flowchart, and the motion detection of 1 × 1 point will be described later.

  After detecting the motion of 1 × 1 point, the offset of the macroblock is calculated (S102). The offset is the amount of movement of the macroblock and is obtained from motion detection at 1 × 1 point. Further, the motion vector of the pixel in the macro block to which the offset is added is obtained from the motion detection result of 1 × 1 point (S103). Thereafter, motion compensation is performed (S104). Since the prediction difference image is considered to have eliminated redundancy between adjacent pixels so that it is not necessary to apply DCT, the prediction difference image is quantized without performing DCT and assigned with variable coding bits to perform the LZ77 method or the range coder. (S105). Also, variable length bits are assigned to the obtained motion vector (S106), and compression is completed by the LZ77 method or a range coder (S110). This is performed for all macroblocks, and the compression of motion compensation predictive coding (P picture) is completed. In the I picture, the color filter output directly compressed by the PNG method, the LZ77 method, a range coder or the like is used.

  Next, the 1 × 1 point motion detection in FIG. 5B will be described. In the 1 × 1 point motion detection, a difference between an image to be compressed and a reference image pixel in the vicinity thereof is found to find a pixel that minimizes the evaluation function expressed by the equation (1).

As for the motion vector, as shown in FIG. 6, the motion vectors of R, G1, G2, and B are searched simultaneously. Therefore, each color has the same motion vector. Further, the reason why the minimum value Err _min is evaluated using the R, G, and B components without using the luminance component is that the minimum value of luminance does not always become the minimum with R, for example. It is. This is because such a false minimum value is generated because the luminance does not include a color component. Here, R indicates a reference image, and R ′ indicates a compression target image. The same applies to G and G ′ and B and B ′. G1 and G2 are obtained by separating Bayer's G into two screens G1 and G2 in FIG. 2B. A unique vector number n is attached to the motion vector. A small vector number n is assigned from a combination of (x, y) having a small absolute value of x and y, and this vector number n is variable-length encoded. The reason why the motions of R, G1, G2, and B are not separately detected is that the motion vector to be compressed increases four times.

  A method for obtaining the macroblock offset (motion vector) and the motion vector of the pixel in the macroblock based on the motion vector obtained by the motion detection of 1 × 1 point will be described (FIGS. 7 and 8). First, the numbers written in the lattice of FIG. 7A are the motion vector (x, y) and the vector number n attached thereto, as shown in FIG. 7C. The numbers in the grid in FIG. 7A are intentional numbers that are easy to understand. Incidentally, the slanted arrow in FIG. 7A indicates the point indicated by the vector of vector number n. In this way, there are many cases where many pixels have similar vectors. As shown in FIG. 7C, the vector number 2 = (2, 2) having the highest frequency is the most frequent, and this is set as the motion vector of the macroblock (OffSetX, OffSetY) = (2, 2 ) Is set. By the way, the macroblock here is 4 × 4 pixels as shown by a thick square in FIG.

  Next, the difference between the 1 × 1 motion vector of FIG. 7A and the vector of (OffSetX, OffSetY) = (2, 2) is taken. When the vector is replaced with the vector number again, the result is as shown in FIG. As seen from this, many zero vectors are included in pixel motion vectors.

  FIG. 8 is a schematic representation of the above description. FIG. 8A shows a pixel motion vector obtained by 1 × 1 motion detection. When the motion vectors of all the pixels indicate a long distance, the efficiency of variable length coding is deteriorated. Therefore, as shown in FIG. 8B, a certain area is collectively handled as a macroblock, and after moving together by the offset, individual pixels inside are moved. By doing so, motion vectors having a large amount of movement are reduced. As a result, the number of zero vectors and the like increase, and the variable length coding efficiency improves.

  In addition, the macroblock motion vector is redundant because it is easy to take the same value between adjacent macroblocks, such as when the entire screen is moving in parallel, such as camera panning. As described above, the compression ratio is improved by separating the motion vector obtained by the 1 × 1 motion detection in FIG. 8A into the motion of the macroblock and the motion of the pixels in the macroblock in FIG. By this method, the compression rate of a moving image with a lot of motion is remarkably increased.

  After the motion vector is obtained, it is necessary to obtain a prediction difference image. The equation shown in Equation 2 obtains a prediction difference image between the reference image and the compression target image, and quantizes it with the quantization coefficient QT (FIG. 5A).

In this equation, R _QT , G _1QT , G _2QT , and B _QT indicate quantized prediction difference images. R, G1, G2, and B indicate reference images, and R ′, G1 ′, G2 ′, and B ′ indicate compression target images. (X _ref , y _ref ) indicates the coordinates of the reference image, and (x _inp , y _inp ) indicates the coordinates of the compression target image. Note that (x _ref , y _ref ) is a coordinate indicating a pixel in a search region of maximum ± MV centered on (x _inp , y _inp ). Here, DCT is not performed. The reason is that motion compensation with sufficient accuracy is performed by motion detection / compensation of 1 × 1 point, and a considerable amount of correlation between adjacent pixels is removed without performing DCT. The scalar quantization used here is a quantization method in which the quantization coefficients are uniformly the same throughout the screen. Since there is no quantization table, processing is simplified. If QT is selected to be a power of 2, a quantization circuit can be configured with only a shift register.

  Next, variable length coding of two types of motion vectors (macroblock motion vector and pixel motion vector in the macroblock) will be described. The motion vector of the macroblock takes a difference from the motion vector of the adjacent macroblock. The same encoding method is used for the motion vector of the macroblock obtained from the difference and the motion vector of the pixel in the macroblock.

  As shown in FIG. 9B, a vector number n is assigned to the motion vector component (x, y) as a pair, and variable length bits are assigned to this n. Short bits are assigned to vector numbers such as (0, 0) and (1, 0) having a high appearance frequency. As shown in FIG. 9A, a binary tree is created so that the smaller vector number is closer to the root of the tree. The variable length bit allocation is obtained by reversing the path from the target node toward the root.

  That is, when 5 is encoded, starting from node 5, if the parent is traced from the left child, 0 is output, and if the parent is traced from the right child, 1 is output. If you get to the root of the tree here, it will be 01. If the output codes are arranged in reverse, it becomes 10 and this is the desired path. When decoding, starting from the root and searching the tree from the left (1) to the right (0) as shown, the target node 5 can be reached.

  In FIG. 10, variable length bits are assigned to vector numbers 8 and 4. In FIG. 10A, in the case of 8, it starts from the node written as 8 toward the root of the binary tree. When transitioning to the parent 3 of 8, since the node 8 is a right child as viewed from the parent 3, 1 is output. Next, transition from 3 to parent 1. Since the child 3 is the left child as seen from the parent 1, 0 is output. Finally, since a transition is made from the left child to the parent root, 0 is output again. These are arranged in reverse, and 001 becomes a path to the node 8.

  FIG. 10B shows auxiliary information for the code 001 obtained in this way. The code 001 needs to specify the length of the code. Otherwise, when binary bit streams are arranged, it is impossible to know where and where the corresponding code is. Therefore, in order to accurately extract 001, another auxiliary stream is generated. This stream is the code shown in FIG. For information 001, FIG. 10 (b) outputs 1 as a cue where 8 begins. Three 0s are output while 3 bits of 001 are output. Therefore, 1000 is output for the path 001. Similarly, in the case of 4, 1 is output as a cue to start the code for the path 01 and 2 and 0 of 2 bits is output for 01 in FIG. 10A. As a result, 100 is output. When a variable length code is assigned in this way, this code is compressed using the LZ77 method or a range coder. This output is output to the multiplexer.

Next, variable-length coding of quantized prediction difference images R _QT , G _1QT , G _2QT , and B _QT will be described (FIG. 5A). The motion-compensated prediction difference image is quantized with a quantizer. Variable length bits are assigned to the quantized prediction difference image. The prediction error is also 0 most frequently, and 1 and −1 are likely to have the next highest appearance frequency. For this reason, as shown in FIG. 11, a binary tree having 0 as the root of the tree is created, and 1, −1, 2, 2, −2,. By creating a binary tree that approaches the root of the tree in ascending order and expressing the path to the node containing the target difference value and its length in binary, variable length is the same as for motion vectors Assign bits.

  As an example, variable length codes are assigned to two data of 4 and 3 as difference values. Starting from the node written as 4 and following the root of the binary tree, the transition from the right child to the parent is repeated three times, so the output is 000. This is reversed and becomes 000. The length of the sign is 3. Similar to the variable length coding of the motion vector described above, the auxiliary stream for cutting out the value from the bit stream outputs 1 as the first cue, followed by three code bits 0 of the next 3 bits. As shown in FIG. 12B, it becomes 1000. Similarly, when 3 enters, 10 in FIG. 12A and 100 in FIG. Thus, in (a), the codes output when transitioning from a certain node to the root are arranged in reverse, and the path from the root of the tree to a certain node (node) is set. Since the codes in FIGS. 12A and 12B include redundancy, the codes are encoded by the LZ77 method, a range coder, or the like. This encoder is not particularly specified, and any encoder can be used as long as it is efficient and has a small hardware.

  As described above, according to the image coding apparatus and the image coding method of the present embodiment, noise is hardly generated in image compression, the hardware scale is small, and the amount of calculation can be suppressed. Specifically, the direct compression of the Bayer image eliminates the need for an interpolation circuit on the camera side, and similarly eliminates a conversion circuit from the RGB color system to the YUV color system. Furthermore, a high-quality difference image can be obtained by adopting both a component indicating a rough motion of the subject and a component indicating a fine motion for motion detection.

  Furthermore, motion detection in units of 1 × 1 pixels can reduce the calculation cost of the sum of absolute differences compared to direct motion detection using N × N pixels. Further, the search area for detecting the motion of the Bayer image is one-fourth that in the case of using a bitmap.

  Further, even without DCT, high-quality compression can be performed, there is no deterioration in image quality related to DCT such as block noise, and hardware and calculation costs required for DCT can be reduced.

  Furthermore, since the quantizer is also a simple scalar quantization, the hardware can be simplified. In addition, since there is no zigzag scan or the like, hardware can be simplified.

  Furthermore, since the Bayer image occupies only one third of the buffer memory of the bitmap image, the buffer memory size that becomes a bottleneck in moving image compression can be greatly reduced.

  As described above, the image encoding device and the image encoding method of the present invention can be applied to a device that transmits an image of a micro camera in addition to a medical device such as a capsule endoscope.

It is a block diagram which shows an example of the image coding apparatus which concerns on this invention. It is explanatory drawing of the color filter of the image coding apparatus. It is explanatory drawing which shows operation | movement of the image coding method. It is a block diagram which shows the image coding method. It is a flowchart which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method. It is explanatory drawing which shows operation | movement of the image coding method.

Explanation of symbols

5 ... Imaging unit 12 ... Vertical scanner 14 ... Noise canceling circuit 16 ... Horizontal scanner 18 ... Amplifier 20 ... A / D converter 24 ... Buff memory 26 ... Prediction Differential encoding circuit

Claims (10)

In an image coding apparatus for a moving image of an image filter primary color filter image that performs coding after extracting a plurality of pixel blocks from a block composed of N × N pixels of an image array using a color filter and separating each of the color components.
Motion detection means for detecting motion at 1 × 1 point in the pixel block and extracting a motion vector;
An image encoding apparatus comprising: compression means for compressing the moving image based on the motion vector.   2. The image coding apparatus according to claim 1, wherein compression of motion compensation prediction coding is used as the compression means.   3. The image coding apparatus according to claim 1, wherein the motion detection unit extracts a motion vector by taking a difference between a pixel in a compression target image and a reference image in the vicinity thereof.   4. The motion vector according to claim 1, wherein the motion vector includes a motion vector of a macro block itself within a predetermined range of the pixel block and a motion vector of a pixel in the macro block. The image encoding device described.   The motion detection unit performs motion detection simultaneously on the color components of the pixel blocks of one red (R), two green (G1, G2), and one blue (B). The image encoding device according to any one of claims 1 to 4. In the image coding method of a moving image of an image filter primary color filter image that performs coding after extracting a plurality of pixel blocks separately for each color component from a block consisting of N × N pixels of an image array using a color filter,
Motion detection is performed on the pixel block at 1 × 1 point to extract a motion vector,
An image encoding method comprising compressing the moving image based on the motion vector.   7. The image encoding method according to claim 6, wherein compression of motion compensated prediction encoding is used as the compression.   8. The image encoding method according to claim 6, wherein the motion detection extracts a motion vector by taking a difference between a pixel in a compression target image and a reference image in the vicinity thereof.   9. The motion vector according to claim 6, wherein the motion vector includes a motion vector of a macroblock itself within a predetermined range of the pixel block and a motion vector of a pixel in the macroblock. The image encoding method described.   7. The motion detection is performed simultaneously for the color components of the pixel blocks of one red (R), two green (G1, G2), and one blue (B). The image encoding method according to claim 9.